A model automatic generation method based on bottle body label quality detection

Abstract

The present application relates to the technical field of automated visual inspection, and discloses a model automatic generation method based on bottle body label quality detection, which comprises the following steps: collecting multi-angle images of the bottle body and carrying out denoising, enhancement and brightness equalization processing; extracting a curved label through a target detection and perspective transformation algorithm; further generating a complete label image without distortion through image registration and fusion technology, and taking the complete label image as a training sample set of a label defect detection model constructed based on a double-branch deep learning network; and finally, according to collected production line feedback data and user feedback data, performing hot updating on the label defect detection model through a closed-loop feedback and continuous learning mechanism. The present application effectively solves the technical problems of imaging deformation of the curved label, difficulty in detecting various defects and performance degradation after model deployment, and realizes high-precision, high-robustness and continuously-evolving automated label quality detection.

Description

Technical Field

[0001] This invention relates to the field of automated visual inspection technology, and in particular to an automatic model generation method based on bottle label quality inspection. Background Technology

[0002] Traditional bottle label quality inspection mainly relies on manual visual inspection or simple machine vision. However, since the bottle body is cylindrical or irregularly shaped, a single-angle camera cannot capture complete label information. Although multi-view imaging can alleviate the problem, how to accurately correct, register and fuse curved surface images from different angles into a complete and distortion-free label image has always been a technical challenge.

[0003] In terms of defect identification models, traditional machine learning methods rely on manually designed features, which are difficult to cope with the diversity and complexity of label defects, such as the difficulty in identifying subtle printing defects and shallow bubbles. While single deep learning models have the ability to learn features on their own, they often cannot simultaneously take into account defect classification and pixel-level precise positioning when dealing with label defects, thus limiting their application in industrial quality inspection for quantitative analysis of defects and process traceability.

[0004] Once deployed, the performance of traditional label defect detection models remains fixed. After being put into production, their performance tends to gradually decline over time, increasing the false detection and false negative rates. The traditional solution is to periodically shut down the entire production line, retrain the model, and then redeploy it. This process is time-consuming, labor-intensive, and costly, making it difficult to meet the needs of flexible manufacturing. Summary of the Invention

[0005] The purpose of this invention is to propose an automatic model generation method based on bottle label quality inspection in order to solve the problems in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an automatic model generation method based on bottle label quality inspection, comprising the following steps:

[0007] Step S1: Capture images of the labeled bottle from multiple angles to obtain the original image;

[0008] Step S2: Denoise, enhance, and equalize the brightness of the original image to obtain a preprocessed image;

[0009] Step S3: Extract the label-containing regions from the preprocessed image using a target detection algorithm, and perform surface correction and multi-view fusion to obtain the label image;

[0010] Step S4: Use image processing algorithms to annotate defects in the labeled images and construct a standardized sample set containing a training set, a validation set, and a test set.

[0011] Step S5: Construct a label defect detection model based on a dual-branch deep learning network, and train the label defect detection model using the training set;

[0012] Step S6: Use the validation set and test set to verify the detection accuracy and generalization ability of the trained label defect detection model;

[0013] Step S7: Deploy the validated label defect detection model to the detection equipment and collect feedback data from the production line;

[0014] Step S8: Design a visual interactive system and collect user feedback data;

[0015] Step S9: Based on production line feedback data and user feedback data, periodically trigger the incremental learning and adjustment process of the label defect detection model.

[0016] Furthermore, step S1 also includes the following sub-steps:

[0017] S1-1, Place the empty bottle to be image acquired into a closed image acquisition dark box, arrange multiple industrial cameras evenly around the bottle, and adjust the incident angle and light intensity of the LED light source according to the bottle material and label material.

[0018] S1-2 uses a rotating clamp to rotate the bottle at a constant angular velocity, simultaneously triggering multiple industrial cameras to capture images.

[0019] S1-3 uses a method based on image sharpness evaluation function and brightness statistics to perform real-time quality assessment of captured images, automatically removes invalid images caused by motion blur, out of focus or overexposure, and encodes the remaining valid images according to timestamp and spatial location to generate structured original images.

[0020] Furthermore, step S2 also includes the following sub-steps:

[0021] S2-1, the original image is denoised by combining Gaussian filtering with nonlocal mean denoising algorithm to obtain a denoised image;

[0022] S2-2, the contrast of the denoised image is enhanced by limiting the contrast adaptive histogram equalization algorithm, and the edge of the denoised image is further sharpened by the multi-scale retinal enhancement algorithm to obtain the detail-enhanced image;

[0023] S2-3, using a gamma correction and color constancy algorithm based on the luminance channel, performs luminance equalization processing on the detail-enhanced image to obtain an equalized image;

[0024] S2-4, Perform color space unification and standard size conversion on the equalized image to obtain a preprocessed image.

[0025] Furthermore, step S3 also includes the following sub-steps:

[0026] S3-1, using a Faster R-CNN-based object detection algorithm, locates and initially outlines the area on the bottle containing the label in the preprocessed image;

[0027] S3-2, using edge detection algorithm and Hough transform, extract the bottle outline and label corner points from the initially defined label area, and further use perspective transformation algorithm to correct the image of the label area into a planar rectangle without geometric deformation, thus obtaining the corrected image;

[0028] S3-3: For multiple corrected images with the same label from different acquisition perspectives, an image registration algorithm based on feature point matching and an image fusion algorithm based on stitch lines are fused together. The fused images are then cropped and their resolution is standardized to obtain the label image.

[0029] Furthermore, step S4 also includes the following sub-steps:

[0030] S4-1, Automatically identify potential defect areas in the label image using a template matching method. These potential defect areas include, but are not limited to, wrinkles, bubbles, stains, curling edges, and printing defects.

[0031] S4-2, record the category and location information of the identified defect areas in the form of bounding boxes or polygons to generate a preliminary annotation file;

[0032] S4-3, the initial labeled files are divided into training set, validation set and test set according to a preset ratio and organized into a unified directory structure to obtain a standardized sample set;

[0033] S4-4, Perform online data augmentation on the training set in the standardized sample set. The online data augmentation includes random rotation, brightness and contrast adjustment, and the addition of Gaussian noise and random occlusion.

[0034] Furthermore, step S5 also includes the following sub-steps:

[0035] S5-1, Construct a label defect detection model based on a dual-branch deep learning network. The dual-branch deep learning network includes a feature extraction branch based on a pre-trained ResNet network and a defect segmentation branch based on a U-Net structure fully convolutional network.

[0036] S5-2 inputs the labeled images in the training set into the feature extraction branch of the label defect detection model and outputs high-level semantic feature maps and low-level detail feature maps.

[0037] S5-3, The feature map is fused at multiple scales using the feature pyramid algorithm, and the fused features are input into the defect segmentation branch to output the defect localization and classification results;

[0038] S5-4, using the AdamW optimizer, the training set is used to iteratively train the label defect detection model, and the model learning rate is dynamically adjusted through a cosine annealing learning rate scheduling strategy until the model loss function converges, thus obtaining the trained label defect detection model.

[0039] Furthermore, step S6 also includes the following sub-steps:

[0040] S6-1 tests the trained label defect detection model using validation and test sets, outputs defect prediction results, and evaluates its detection accuracy.

[0041] S6-2 uses external independent test sets collected from different production lines, bottle types and lighting conditions to test the label defect detection model and evaluate its generalization ability index.

[0042] S6-3, Generate the confusion matrix of the label defect detection model on the validation set and perform error analysis. The error analysis includes statistical analysis of false detections and false negatives of various defects, as well as locating the main sources of errors.

[0043] S6-4. Based on the evaluated detection accuracy and generalization ability, as well as the results of error analysis, determine whether the label defect detection model meets the standards. If it does, proceed to the next step of deployment. If it does not meet the standards, perform defect pattern attribution analysis and optimize the structure of the training sample set or the label defect detection model in a targeted manner. Repeat the above verification process for the optimized label defect detection model.

[0044] Furthermore, step S7 also includes the following sub-steps:

[0045] S7-1 uses the TensorRT toolchain to perform structural transformation on the verified label defect detection model, encapsulates the transformed label defect detection model into an independent inference service module, and integrates it into the vision inspection system of the actual production line through the API interface.

[0046] S7-2, Establish a data recording and feedback mechanism in the visual inspection system to automatically save the original image of each bottle, model inference results, inspection timestamps and pass / fail judgment results;

[0047] S7-3, bottles identified as defective by the label defect detection model are transported to the re-inspection station, where quality inspectors manually confirm the defect. The results of the manual re-inspection are then used as truth labels and stored in association with the model inference results to form production line feedback data with correction labels.

[0048] Furthermore, step S8 also includes the following sub-steps:

[0049] S8-1, construct a visual interactive system that includes a real-time detection screen display area, a defect result list area, and a manual judgment interaction area;

[0050] S8-2, The manual judgment interaction area provides operable virtual buttons including "confirm", "false alarm", "missed alarm" and "classification correction". When the user operates on the detection results of the label defect detection model through the virtual buttons, the system automatically records the current image data of the bottle, the original output of the model, the user operation results and the operation timestamp.

[0051] S8-3, In the manual judgment interaction area, a labeling tool is provided to the user. When the model has a missed detection, the user can manually label the defect area through the labeling tool, and the system automatically records the manual labeling results.

[0052] S8-4, The image data, the original output of the model, the user operation results, the operation timestamp, and the manual annotation results are packaged into user feedback data.

[0053] Furthermore, step S9 also includes the following sub-steps:

[0054] S9-1: Periodically extract newly added production line feedback data and user feedback data, and perform deduplication, validity verification and consistency verification to form an incremental dataset;

[0055] S9-2 mixes the incremental dataset with the original training sample set to form a new generation training set, and uses knowledge distillation technology to incrementally learn and train the weights of the deployed label defect detection model.

[0056] S9-3 performs version management on the new label defect detection model obtained after incremental learning training, and conducts parallel A / B testing with the online old label defect detection model in shadow mode to compare its performance and stability.

[0057] S9-4 When the new label defect detection model outperforms the old label defect detection model in A / B testing and the key indicators are stable, the old label defect detection model on the line is replaced by hot update technology.

[0058] The beneficial effects of the technical solution provided by this invention include at least the following:

[0059] This invention achieves high-quality, full-view digital reconstruction of bottle labels through multi-view label image extraction and fusion technology, and finally generates a high-definition label image with no geometric deformation and complete information, fundamentally solving the problem of curved surface label imaging and significantly improving the basic data quality and reliability of defect identification.

[0060] This invention combines the powerful feature extraction capabilities of ResNet with the precise pixel-level segmentation capabilities of U-Net to design a label defect detection model based on a dual-branch deep learning network structure. This model enables end-to-end fine-grained detection of various defects such as wrinkles, bubbles, and dirt, significantly improving the accuracy and practicality of the detection.

[0061] This invention introduces a closed-loop feedback and continuous learning mechanism through production line re-inspection and user interaction, enabling the label defect detection model to continuously self-optimize in production. This allows the system to adapt to production line changes, absorb human experience, and cope with new defects, ensuring the stability and advancement of long-term application.

[0062] In summary, this invention systematically integrates advanced image processing, deep learning, and feedback loop systems to form a complete intelligent detection solution that combines perception, decision-making, and optimization. This solution enhances the intelligence level and adaptability of the entire production line and provides strong technical support for the digital transformation of industrial quality inspection. Attached Figure Description

[0063] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a flowchart of a method provided in an embodiment of the present invention. Detailed Implementation

[0065] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an automatic model generation method based on bottle label quality inspection proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0067] The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0068] The following description, in conjunction with the accompanying drawings, details the specific scheme of the automatic model generation method based on bottle label quality inspection provided by this invention.

[0069] Please see Figure 1 The diagram illustrates a method flowchart for automatically generating a model based on bottle label quality inspection, according to an embodiment of the present invention. The method includes the following steps:

[0070] Step S1: Capture images of the labeled bottle from multiple angles to obtain the original image;

[0071] Step S1 further includes the following sub-steps:

[0072] S1-1, Place the empty bottle to be image acquired into a closed image acquisition dark box, arrange multiple industrial cameras evenly around the bottle, and adjust the incident angle and light intensity of the LED light source according to the bottle material and label material.

[0073] S1-2 uses a rotating clamp to rotate the bottle at a constant angular velocity, simultaneously triggering multiple industrial cameras to capture images.

[0074] S1-3 uses a method based on image sharpness evaluation function and brightness statistics to perform real-time quality assessment of captured images, automatically removes invalid images caused by motion blur, out of focus or overexposure, and encodes the remaining valid images according to timestamp and spatial location to generate structured original images.

[0075] It should be noted that a closed image acquisition dark box is an image acquisition environment specifically designed for industrial visual inspection. It typically uses a low-reflectivity black light-absorbing material to cover the inner walls to reduce the impact of light reflection and scattering on image quality, providing high-quality raw image data for subsequent image processing and defect detection.

[0076] Image sharpness evaluation functions typically use gradient-based calculations to measure the variation in pixel grayscale values ​​in an image, thereby quantifying the sharpness of image edges and identifying blurred images caused by motion blur or defocus. Meanwhile, brightness statistics methods analyze the overall brightness distribution of an image and, in conjunction with a preset brightness threshold range, identify image failures caused by underexposure or overexposure. This facilitates subsequent image registration, fusion, and source analysis, improving the automation level and reliability of the overall detection process.

[0077] Step S2: Denoise, enhance, and equalize the brightness of the original image to obtain a preprocessed image;

[0078] Step S2 further includes the following sub-steps:

[0079] S2-1, the original image is denoised by combining Gaussian filtering with nonlocal mean denoising algorithm to obtain a denoised image;

[0080] S2-2, the contrast of the denoised image is enhanced by limiting the contrast adaptive histogram equalization algorithm, and the edge of the denoised image is further sharpened by the multi-scale retinal enhancement algorithm to obtain the detail-enhanced image;

[0081] S2-3, using a gamma correction and color constancy algorithm based on the luminance channel, performs luminance equalization processing on the detail-enhanced image to obtain an equalized image;

[0082] S2-4 performs color space unification and standard size conversion on the equalized image to obtain the preprocessed image.

[0083] It should be noted that Gaussian filtering is a linear smoothing filter that uses convolution operations based on Gaussian kernel functions to perform weighted averaging on the image, effectively filtering out high-frequency noise in the image. However, a single Gaussian filtering method may cause blurring of image edges. Therefore, a nonlocal mean denoising algorithm is further introduced. It utilizes the redundant information in the image and calculates the similarity between all pixel blocks in the image to perform a similarity-based weighted average on the target pixels. This smooths noise while preserving the texture details and edge structure of the image. The combination of the two achieves collaborative denoising from local smoothing to global search, providing a high-quality denoised image foundation for subsequent image enhancement steps.

[0084] Compared to a single global histogram equalization method, the contrast-limited adaptive histogram equalization algorithm improves upon the problem of excessive enhancement in local areas and noise amplification. Specifically, it involves: first, dividing the input image into several small, continuous regions; then, performing histogram equalization independently on each region; and finally, redistributing pixels exceeding a set threshold across the entire histogram range by cropping the histogram amplitude, thereby suppressing noise amplification. Bilinear interpolation is then used to smoothly stitch the equalization results of each region together to eliminate artifacts at region boundaries. This method is particularly suitable for label images containing both bright and dark areas, significantly improving the visual separability of defects such as wrinkles and bubbles from the image background.

[0085] Gamma correction adjusts the grayscale response curve of an image by nonlinearly transforming the image's brightness channel, effectively expanding or compressing the image's dynamic range and achieving overall brightness equalization. Meanwhile, the color constancy algorithm aims to eliminate the influence of light source color on the inherent color of objects and correct color deviations caused by illumination color temperature. The combination of the two ensures that the subsequently processed image is not only uniform in brightness distribution but also accurate and consistent in color, laying a stable foundation for the accurate identification and classification of defects.

[0086] Step S3: Extract the label-containing regions from the preprocessed image using a target detection algorithm, and perform surface correction and multi-view fusion to obtain the label image;

[0087] Step S3 further includes the following sub-steps:

[0088] S3-1, using a Faster R-CNN-based object detection algorithm, locates and initially outlines the area on the bottle containing the label in the preprocessed image;

[0089] S3-2, using edge detection algorithm and Hough transform, extract the bottle outline and label corner points from the initially defined label area, and further use perspective transformation algorithm to correct the image of the label area into a planar rectangle without geometric deformation, thus obtaining the corrected image;

[0090] S3-3: For multiple corrected images with the same label from different acquisition perspectives, an image registration algorithm based on feature point matching and an image fusion algorithm based on stitch lines are fused together. The fused images are then cropped and their resolution is standardized to obtain the label image.

[0091] It should be noted that the object detection algorithm based on Faster R-CNN is used to accurately locate and initially define the label region from the complex bottle background. Specifically, it includes: firstly, extracting the shared feature map of the preprocessed image through a deep convolutional network; then, quickly determining whether the region contains the target (i.e., whether the image contains the label) and initially regressing the bounding box coordinates through anchor points of various preset scales and aspect ratios, outputting a series of candidate regions; finally, fixing the feature map size of each candidate region through the RoI pooling layer and feeding it into the classification and regression layers. This mechanism can effectively address the robust detection problem of bottle labels at different angles, scales, and with partial occlusion.

[0092] Edge detection algorithms, together with Hough transform, are used to accurately extract the bottle outline and label corner points from the initially defined label region. Specifically, the algorithm first extracts all significant edges representing the object boundary in the label region using edge detection. Then, Hough transform is used to detect specific geometric shapes from these edges. For the bottle outline, Hough circle transform may be used to detect the circular cross-section of the bottle, while for the label corner points, Hough line transform is used to detect the boundary lines of the label region, and the corner point position is determined by calculating the intersection of the lines. This method has good fault tolerance and can stably extract the key geometric features of the bottle and label from complex images that may contain interference.

[0093] Image registration algorithms based on feature point matching aim to accurately align multiple corrected images with the same label from different viewpoints in space. Specifically, the algorithm involves: first, detecting local feature points invariant to image rotation, scaling, and brightness changes in each image; calculating the Euclidean distance between these feature points to establish matching pairs between feature points in different images; then, using a robust estimation algorithm to estimate the homography matrix describing the perspective transformation relationship between the image to be registered and the reference image from the matching pairs that may contain mismatches; and finally, resampling and transforming the image to be registered using this matrix to accurately align it in space with the reference image.

[0094] Since images from different perspectives may differ in brightness and color, direct superposition will result in obvious seams. Therefore, an image fusion algorithm based on the seam line is needed to synthesize multiple registered label images into a single, visually seamless, high-quality label image. The core idea of ​​this algorithm is to find an optimal boundary that minimizes the differences between the images on either side of this boundary during stitching. The algorithm typically calculates the difference map between two overlapping images and uses dynamic programming or graph cut methods to find a path with the minimum cumulative difference in the difference map. This path is the optimal seam line. During fusion, pixels on one side of the seam line come from one image, and pixels on the other side come from the other image, thereby minimizing stitching artifacts and achieving a natural transition and smooth integration of multi-view image information.

[0095] Step S4: Use image processing algorithms to annotate defects in the labeled images and construct a standardized sample set containing a training set, a validation set, and a test set.

[0096] Step S4 further includes the following sub-steps:

[0097] S4-1, automatically identifies potential defect areas in label images using template matching methods. Potential defect areas include, but are not limited to, wrinkles, bubbles, stains, curling edges, and printing defects.

[0098] S4-2, record the category and location information of the identified defect areas in the form of bounding boxes or polygons to generate a preliminary annotation file;

[0099] S4-3, the initial labeled files are divided into training set, validation set and test set according to a preset ratio and organized into a unified directory structure to obtain a standardized sample set;

[0100] S4-4 performs online data augmentation on the training set in the standardized sample set. The online data augmentation includes random rotation, brightness and contrast adjustment, and the addition of Gaussian noise and random occlusion.

[0101] It should be noted that the template matching method, as an efficient initial defect localization technique, works by sliding a predefined template representing a defect-free standard region onto the label image to be inspected. The similarity between the template and various regions of the image to be inspected is calculated using the squared difference or correlation coefficient matching method. This allows for the rapid localization of potential defect regions that differ significantly from the standard template. This method can quickly filter out regions with obvious anomalies such as wrinkles, bubbles, stains, or printing defects from a large number of images, providing clear candidate targets for subsequent manual or semi-automatic fine annotation and significantly improving the efficiency of annotation work.

[0102] Dividing the initial labeled files into training, validation, and test sets is a crucial step in model building. Typically, these sets are randomly divided according to a certain ratio, ensuring that all types of defects are evenly distributed across the resulting sets. The training set is the primary dataset used by the model to learn defect features and update weights. The validation set is not used in training but is used to periodically evaluate model performance during training, serving as a basis for adjusting hyperparameters and determining when to stop early. The test set is used to evaluate the model's generalization performance on unknown data after training is complete.

[0103] Step S5: Construct a label defect detection model based on a dual-branch deep learning network, and train the label defect detection model using the training set;

[0104] Step S5 further includes the following sub-steps:

[0105] S5-1, Construct a labeled defect detection model based on a dual-branch deep learning network. The dual-branch deep learning network includes a feature extraction branch based on a pre-trained ResNet network and a defect segmentation branch based on a U-Net structure fully convolutional network.

[0106] S5-2 inputs the labeled images in the training set into the feature extraction branch of the label defect detection model and outputs high-level semantic feature maps and low-level detail feature maps.

[0107] S5-3: The feature map is fused at multiple scales using the feature pyramid algorithm. The fused features are then input into the defect segmentation branch, and the defect localization and classification results are output.

[0108] In S5-4, the AdamW optimizer is used to iteratively train the label defect detection model using the training set. The model learning rate is dynamically adjusted using a cosine annealing learning rate scheduling strategy until the model loss function converges, resulting in a trained label defect detection model.

[0109] It should be noted that the feature extraction branch uses ResNet pre-trained on a large dataset as the backbone network, which is responsible for extracting rich and discriminative features from the input label images. By introducing "residual blocks" and "shortcut connections" structures, it effectively alleviates the gradient vanishing and network degradation problems in deep networks, provides excellent initial parameters for the label defect detection model, and enables it to recognize general visual patterns such as edges, textures and shapes, thus providing a solid information foundation for subsequent defect segmentation.

[0110] The defect segmentation branch adopts a fully convolutional encoder-decoder network based on the U-Net architecture. Its core task is to achieve accurate pixel-level localization and classification of defects. The encoder part gradually downsamples through pooling layers to capture the contextual information of the image, while the decoder part gradually restores the spatial resolution through upsampling operations to achieve accurate localization. U-Net concatenates the high-resolution feature maps of each level of the encoder with the corresponding feature maps of the intermediate layers of the decoder in the channel dimension, thereby combining the detailed information captured by the encoder with the contextual information restored by the decoder, which can clearly identify the boundary of the defect while locating the defect region.

[0111] Feature pyramids are an effective structure for handling multi-scale target problems. In this invention, they are responsible for fusing feature maps from different levels of the ResNet feature extraction branch, which have different spatial resolutions and semantic intensities. Shallow feature maps have high resolution and rich details but weak semantics, while deep feature maps have strong semantics but low resolution and lose details. By constructing a multi-scale feature pyramid with high resolution and strong semantic information, we can simultaneously use the fine details of shallow features to locate small defects and use the abstract semantics of deep features to identify large defects, which significantly improves the ability of the labeled defect detection model to detect defects of different sizes.

[0112] The AdamW optimizer is an improved version of the Adam optimizer. In the original Adam, weight decay is usually coupled with the gradient, which can sometimes interfere with the effect of adaptive learning rate, resulting in poor regularization and overfitting. The improved AdamW can separate the weight decay term and apply it independently when updating the weights, thus making the training process of the label defect detection model more stable and the generalization ability stronger.

[0113] Cosine annealing is a strategy for dynamically adjusting the learning rate. Specifically, it involves setting a relatively high initial learning rate at the beginning of training. As the number of training iterations increases, the learning rate smoothly decreases from its maximum value to a near-zero minimum value according to the cosine function curve. Compared with the traditional step-down strategy, this strategy allows the label defect detection model to boldly cross local minima in the early stages of training and finely converge to a better local or global optimum in the later stages of training, thereby achieving better performance.

[0114] Step S6: Use the validation set and test set to verify the detection accuracy and generalization ability of the trained label defect detection model;

[0115] Step S6 further includes the following sub-steps:

[0116] S6-1 tests the trained label defect detection model using validation and test sets, outputs defect prediction results, and evaluates its detection accuracy.

[0117] S6-2 uses external independent test sets collected from different production lines, bottle types and lighting conditions to test the label defect detection model and evaluate its generalization ability index.

[0118] S6-3, generate the confusion matrix of the label defect detection model on the validation set and perform error analysis. Error analysis includes statistical analysis of false positives and false negatives for various types of defects, as well as locating the main sources of errors;

[0119] S6-4. Based on the evaluated detection accuracy and generalization ability, as well as the results of error analysis, determine whether the label defect detection model meets the standards. If it does, proceed to the next step of deployment. If it does not meet the standards, perform defect pattern attribution analysis and optimize the structure of the training sample set or the label defect detection model in a targeted manner. Repeat the above verification process for the optimized label defect detection model.

[0120] It should be noted that the confusion matrix is ​​used to visually present the correspondence between the prediction results of the label defect detection model and the real labels. The rows of the matrix represent the real defect categories, the columns represent the predicted categories, the values ​​on the main diagonal represent the number of correctly classified samples, and the values ​​off the diagonal reveal the various errors made by the model, such as false positives and false negatives. By analyzing the confusion matrix, key indicators such as precision, recall, and overall accuracy for each type of defect can be calculated, thereby identifying which specific defect types the label defect detection model is inadequate in, providing clear data support and directional guidance for subsequent optimization.

[0121] Table 1 provides a specific example of how to determine whether a label defect detection model meets the standards:

[0122] Table 1. Comprehensive Evaluation Table of Label Defect Detection Model

[0123]

[0124] Defect pattern attribution analysis is a systematic diagnostic process conducted to locate the root cause when the performance of a labeled defect detection model is substandard. The analysis process typically includes: first, data-level attribution, checking whether there are too few samples of certain defect categories or poor labeling quality in the training data, or whether there is a lack of a specific defect morphology or background interference; second, model-level attribution, analyzing whether the labeled defect detection model overfits to certain non-critical features, or whether the receptive field is insufficient to capture the global contextual information of large-sized defects, thereby determining whether the problem stems from data quality, data distribution, or the model structure itself.

[0125] Step S7: Deploy the validated label defect detection model to the detection equipment and collect feedback data from the production line;

[0126] Step S7 further includes the following sub-steps:

[0127] S7-1 uses the TensorRT toolchain to perform structural transformation on the verified label defect detection model, encapsulates the transformed label defect detection model into an independent inference service module, and integrates it into the vision inspection system of the actual production line through the API interface.

[0128] S7-2, Establish a data recording and feedback mechanism in the visual inspection system to automatically save the original image of each bottle, model inference results, inspection timestamps and pass / fail judgment results;

[0129] S7-3, bottles identified as defective by the label defect detection model are transported to the re-inspection station, where quality inspectors manually confirm the defect. The results of the manual re-inspection are then used as truth labels and stored in association with the model inference results to form production line feedback data with correction labels.

[0130] It should be noted that the verified label defect detection model is processed through the TensorRT toolchain. Specifically, the label defect detection model is first converted into TensorRT's proprietary format. Then, the converted model is adjusted by layer fusion, accuracy calibration, and automatic kernel tuning to reduce computation and memory overhead. This allows the model to fully utilize the computing resources of edge-deployed devices and meet the stringent real-time requirements of production line inspection.

[0131] An API (Application Programming Interface) refers to a predefined set of functions, protocols, and tools used to enable communication and integration between different software modules. It is used to decouple and integrate the independent inference service module, which is optimized and encapsulated by TensorRT, with the existing vision inspection system on the production line. After capturing an image of the bottle, the vision inspection system does not need to worry about the complex calculations inside the model. It only needs to send the image data to the API, and the inference service module calls the optimized model in the background to complete the defect detection and return the inference results to the vision inspection system. This API-based integration method greatly improves the modularity, maintainability, and scalability of the system.

[0132] Step S8: Design a visual interactive system and collect user feedback data;

[0133] Step S8 further includes the following sub-steps:

[0134] S8-1, construct a visual interactive system that includes a real-time detection screen display area, a defect result list area, and a manual judgment interaction area;

[0135] S8-2 provides operable virtual buttons in the manual judgment interaction area, including "Confirm", "False Alarm", "Mistake", and "Classification Correction". When the user operates on the detection results of the label defect detection model through the virtual buttons, the system automatically records the current image data of the bottle, the original output of the model, the user's operation results, and the operation timestamp.

[0136] S8-3 provides users with annotation tools in the manual judgment interaction area. When the model misses a detection, the user can manually annotate the defective area through the annotation tools, and the system will automatically record the manual annotation results.

[0137] S8-4 packages image data, original model output, user operation results, operation timestamps, and manually labeled results into user feedback data.

[0138] It should be noted that the real-time detection display area is one of the core visualization components of the visualization interaction system. Its main function is to synchronously and with low latency present the original bottle images captured by the production line camera and the real-time inference results of the label defect detection model. This area not only displays high-definition bottle images, but also highlights various defect areas detected by the model with bounding boxes of different colors and line widths, linking the abstract output of the model with specific visual images, allowing users to intuitively understand the quality inspection status of the bottle label and the basis for the model's judgment.

[0139] The defect results list area is one of the core visualization components of the interactive visualization system. It is usually located on one side of the interface and lists or tables clearly listing each bottle record that has been judged to have defects by the model in the order of inspection time or batch. This provides a systematic overview and quick search function. Quality inspectors can quickly understand the recent defect distribution of the production line by simply browsing this list, which effectively improves the management efficiency and traceability of the quality inspection process.

[0140] Step S9: Based on production line feedback data and user feedback data, periodically trigger the incremental learning and adjustment process of the label defect detection model;

[0141] Step S9 further includes the following sub-steps:

[0142] S9-1: Periodically extract newly added production line feedback data and user feedback data, and perform deduplication, validity verification and consistency verification to form an incremental dataset;

[0143] S9-2 mixes the incremental dataset with the original training sample set to form a new generation training set, and uses knowledge distillation technology to incrementally learn and train the weights of the deployed label defect detection model.

[0144] S9-3 performs version management on the new label defect detection model obtained after incremental learning training, and conducts parallel A / B testing with the online old label defect detection model in shadow mode to compare its performance and stability.

[0145] S9-4 When the new label defect detection model outperforms the old label defect detection model in A / B testing and the key indicators are stable, the old label defect detection model on the line is replaced by hot update technology.

[0146] It should be noted that knowledge distillation is a model compression and knowledge transfer technique used to enable a new model to learn from the incremental dataset of the old model and its output probability distribution. This guides the new model to generalize more effectively, effectively avoiding the problem of catastrophic forgetting caused by retraining directly on a mixed dataset of old and new models, and ensuring a robust improvement in model performance.

[0147] Parallel A / B testing is a canary release strategy that safely and scientifically evaluates the performance of a new model in a real production environment. Specifically, it involves running an old model (Version A) that is stable online and in the model deployment environment at the same time, and a new model (Version B) that is about to be launched. Both models process real-time data from the production line in parallel. The inference results of the old model continue to be used as the actual basis for production line testing, while the inference results of the new model are only run and recorded in shadow mode. The system synchronously collects and compares the inference results of the two models on the same batch of real data, and compares their performance from multiple dimensions such as detection accuracy, generalization ability, inference speed and stability, providing an important basis for the final decision on whether to fully replace the old model.

[0148] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for automatically generating a model based on bottle body label quality detection, characterized in that, The method comprises: Step S1, multi-angle image acquisition is performed on the bottle body with the label attached, to obtain an original image; Step S2, denoising, enhancement and brightness equalization processing are performed on the original image, to obtain a pretreated image; Step S3, a target detection algorithm is used to extract a region containing the label in the pretreated image, and surface correction and multi-view fusion are performed, to obtain a label image; Step S4, a defect labeling is performed on the label image by using an image processing algorithm, and a standardized sample set comprising a training set, a verification set and a test set is constructed; Step S5, a label defect detection model based on a double-branch deep learning network is constructed, and the label defect detection model is trained by using the training set; In step S5, the following substep is further included: S5-1, a label defect detection model based on a double-branch deep learning network is constructed, the double-branch deep learning network comprising a feature extraction branch based on a pre-trained ResNet network and a defect segmentation branch based on a U-Net structure full convolution network; Step S6, the detection accuracy and generalization ability of the trained label defect detection model are verified by using the verification set and the test set; Step S7, the label defect detection model that passes the verification is deployed to a detection device, and production line feedback data is collected; Step S8, a visual interactive system is designed, and user feedback data is collected; Step S9, according to the production line feedback data and the user feedback data, an incremental learning and adjustment process of the label defect detection model is triggered regularly; In step S9, the following substeps are further included: S9-1, newly added production line feedback data and user feedback data are extracted regularly, and are subjected to deduplication, validity verification and consistency verification, to form an incremental data set; S9-2, the incremental data set is mixed with the original training sample set, to form a new generation of training set, and the weight of the deployed label defect detection model is subjected to incremental learning training by using a knowledge distillation technology; S9-3, the new label defect detection model obtained after the incremental learning training is subjected to version management, and is subjected to parallel A / B testing with an old label defect detection model on line in a shadow mode, and the performance and stability thereof are compared; S9-4, when the performance of the new label defect detection model is better than that of the old label defect detection model in the A / B testing, and the key indicators are stable, the old label defect detection model on line is replaced by using a hot update technology.

2. The model automatic generation method based on bottle body label quality detection according to claim 1, wherein in step S1, the following substep is further included: S1-1, the empty bottle to be subjected to image acquisition is placed in a closed image acquisition dark box, a plurality of industrial cameras are arranged uniformly around the bottle body, and the incident angle and illumination intensity of the LED light source are adjusted according to the bottle body material and the label material; S1-2, the bottle body is driven to rotate at a constant angular velocity by a rotating clamp, and the plurality of industrial cameras are triggered synchronously to capture images. ​ S1-3, automatically removing invalid images caused by motion blur, defocus or overexposure by real-time quality evaluation of captured images based on image sharpness evaluation function and brightness statistics, and encoding the remaining valid images according to timestamp and spatial position to generate structured original images.

3. The method of claim 1, wherein the step S2 further comprises the following sub-steps: S2-1, denoising the original image by a Gaussian filter combined with a non-local means denoising algorithm to obtain a denoised image; S2-2, enhancing the contrast of the denoised image by a limited contrast adaptive histogram equalization algorithm, and further sharpening the edges of the denoised image by a multi-scale retinal enhancement algorithm to obtain a detail-enhanced image; S2-3, performing brightness equalization on the detail-enhanced image by a brightness channel-based gamma correction and color constancy algorithm to obtain an equalized image; S2-4, performing color space unification and standard size conversion on the equalized image to obtain a preprocessed image.

4. The method of claim 1, wherein the step S3 further comprises the following sub-steps: S3-1, locating and preliminarily framing the label-containing area of the bottle body in the preprocessed image by a target detection algorithm based on Faster R-CNN; S3-2, extracting the bottle body contour and label corner points from the preliminarily framed label area by an edge detection algorithm and Hough transform, and further correcting the image of the label area to a planar rectangular image without geometric distortion by a perspective transform algorithm to obtain a corrected image; S3-3, fusing multiple corrected images from different collection angles of the same label by an image registration algorithm based on feature point matching and an image fusion algorithm based on stitching lines, and further cropping and standardizing the resolution of the fused image to obtain a label image.

5. The method of claim 1, wherein the step S4 further comprises the following sub-steps: S4-1, automatically identifying potential defect areas in the label image by a template matching method, wherein the potential defect areas include but are not limited to wrinkles, bubbles, stains, edge lifting and printing defects; S4-2, recording the class and position information of the identified defect areas in the form of a bounding box or a polygon to generate a preliminary annotation file; S4-3, dividing the preliminary annotation file into a training set, a validation set and a test set according to a predetermined proportion and organizing them into a unified directory structure to obtain a standardized sample set; S4-4, performing online data augmentation on the training set in the standardized sample set, wherein the online data augmentation includes random rotation, brightness and contrast adjustment, and adding Gaussian noise and random occlusion.

6. The method of claim 1, wherein the step S5 further comprises the following sub-steps: ​ ​ ​ ​ S5-2, input the label image in the training set into the feature extraction branch of the label defect detection model, output high-level semantic feature maps and low-level detail feature maps; S5-3, perform multi-scale fusion on the feature maps through a feature pyramid algorithm, input the fused features into the defect segmentation branch, and output defect positioning and classification results; S5-4, continue to use the training set to iteratively train the label defect detection model through an AdamW optimizer, and dynamically adjust the model learning rate through a cosine annealing learning rate scheduling strategy until the model loss function converges, thereby obtaining the trained label defect detection model.

7. The model automatic generation method based on bottle body label quality detection according to claim 1, wherein in step S6, the following sub-steps are further included: S6-1, verify the trained label defect detection model through the verification set and the test set, output defect prediction results, and evaluate the detection accuracy; S6-2, use external independent test sets collected from different production lines, different bottle types, and different lighting conditions to test the label defect detection model, and evaluate its generalization ability index; S6-3, generate a confusion matrix of the label defect detection model on the verification set, and perform error analysis, which includes statistics of false positives and false negatives of each type of defect, and locating the main error sources; S6-4, according to the evaluation of the detection accuracy and the generalization ability, and the results of the error analysis, determine whether the label defect detection model meets the standard, if it meets the standard, prepare for the next step of deployment, if it does not meet the standard, perform defect mode attribution analysis, and perform targeted optimization on the training sample set or the structure of the label defect detection model, and repeat the above verification process on the targeted optimized label defect detection model.

8. The model automatic generation method based on bottle body label quality detection according to claim 1, wherein in step S7, the following sub-steps are further included: S7-1, convert the structure of the verified label defect detection model through the TensorRT tool chain, encapsulate the converted label defect detection model into an independent inference service module, and integrate it into the actual production line visual detection system through the API interface set; S7-2, establish a data recording and feedback mechanism in the visual detection system, automatically save the original image of each bottle body, the model inference result, the detection timestamp, and the eligibility determination result; S7-3, convey the bottle body determined as defective by the label defect detection model to the re-inspection station, and manually confirm it by the quality inspection personnel, and store the manual re-inspection result as the true value label and the model inference result in association, forming the production line feedback data with the corrected label.

9. The model automatic generation method based on bottle body label quality detection according to claim 1, wherein in step S8, the following sub-steps are further included: S8-1, build a visual interactive system including a real-time detection picture display area, a defect result list area, and a manual judgment interaction area; ​ ​ ​ S8-2, providing operable virtual buttons including "confirmation", "false positive", "false negative" and "classification correction" in the artificial judgment interaction area, when the user operates the detection result of the label defect detection model through the virtual buttons, the system automatically records the image data of the current bottle body, the model original output, the user operation result and the operation timestamp; S8-3, providing a labeling tool to the user in the artificial judgment interaction area, when the model misses detection, the user can manually label the defect area through the labeling tool, and the system automatically records the manual labeling result; S8-4, packing the image data, the model original output, the user operation result, the operation timestamp and the manual labeling result into user feedback data.

Images