Production line packaging quality monitoring method and system based on AR glasses

Abstract

The invention relates to a production line packaging quality monitoring method and system based on AR glasses, and belongs to the technical field of packaging quality detection.The quality monitoring method comprises the steps that monitoring image data of a to-be-detected packaging box on a packaging production line is obtained in real time and subjected to preprocessing and feature extraction, and a feature vector corresponding to each image is obtained; respectively inputting the images into a pre-constructed packet quality detection model for detection to obtain a packet quality detection result corresponding to each image; calculating a package integrity score of the to-be-tested package box; judging whether the packet integrity score is lower than a preset threshold value or not, and if yes, generating a defect detection report of the to-be-detected packaging box and sending the defect detection report to AR glasses worn by the current operator; and marking the package defect position and the package defect category of the to-be-detected package box according to the defect detection report to obtain defect marking information, and projecting the defect marking information to the view of the current operator through AR glasses. According to the invention, efficient monitoring and intelligent feedback of the packaging quality can be realized.

Description

Technical Field

[0001] The present application relates to the technical field of packaging quality detection, and in particular to a packaging quality monitoring method and system for a production line based on AR glasses. Background Art

[0002] In the field of modern industrial production, packaging quality control is of great significance to product performance and production line efficiency. Especially in high-speed and high-precision production lines, packaging quality defects, such as uneven edges and loose seals, may cause functional problems in the subsequent transportation and use of products, affecting product quality and customer satisfaction. Therefore, how to achieve comprehensive, real-time and accurate detection of packaging quality has become a key requirement in industrial production quality control.

[0003] Common package quality inspection methods usually rely on manual visual inspection or fixed inspection equipment, which have significant limitations. Manual inspection is inefficient and easily affected by human fatigue and subjective judgment, resulting in unstable test results; and although fixed inspection equipment can partially replace manual work, due to its limited viewing angle, it can only perform single or limited range inspections on the packaging box, making it difficult to achieve full coverage. In addition, these inspection methods are usually limited to the problem discovery stage, making it difficult to form a rapid response mechanism.

[0004] At present, with the development of intelligent technology, the shortcomings of traditional detection methods are becoming more and more obvious. The production line has put forward higher requirements on the real-time and accuracy of detection equipment, and at the same time, it is necessary to realize the instant feedback of detection data to form a closed-loop process from defect detection to operation repair. Therefore, how to achieve efficient monitoring and intelligent feedback of package quality has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to achieve efficient monitoring and intelligent feedback of packaging quality, the present application provides a production line packaging quality monitoring method and system based on AR glasses.

[0006] In the first aspect, the present application provides a production line packaging quality monitoring method based on AR glasses, which adopts the following technical solution: A packaging quality monitoring method for a production line based on AR glasses, the quality monitoring method comprising: Acquire monitoring image data of the packaging box to be tested on the packaging production line in real time; wherein the monitoring image data includes images of different parts of the packaging box to be tested collected from multiple angles; Preprocessing and feature extraction are performed on the monitoring image data to obtain a feature vector corresponding to each image; Input the feature vectors corresponding to each image into the pre-built packet quality detection model for detection, and obtain the packet quality detection results corresponding to each image; Based on a weighted fusion algorithm, a package integrity score of the package box to be tested is calculated according to the package quality detection result corresponding to each image; Determine whether the package integrity score is lower than a preset threshold, and if so, generate a defect detection report of the package box to be tested; and send the defect detection report to the AR glasses worn by the current operator; Marking the package defect position and package defect category of the package box to be tested according to the defect detection report to obtain defect marking information; The defect marking information is projected into the field of view of the current operator through the AR glasses.

[0007] By adopting the above technical solutions, combined with multi-camera image acquisition and AI detection algorithms, comprehensive intelligent detection of package quality is achieved, and defect information is fed back to the operator in real time through AR glasses, forming a closed-loop workflow from defect detection to repair guidance, so that package quality problems on the production line can be discovered and intuitively presented in real time, which not only improves the accuracy and efficiency of detection, but also significantly reduces the errors that may be caused by human intervention, thereby comprehensively optimizing the production quality and management level of the packaging production line, and providing innovative solutions for intelligent industrial production lines.

[0008] Optionally, the step of preprocessing and extracting features from the monitoring image data to obtain a feature vector corresponding to each image includes: Performing image denoising on the monitoring image data; Perform edge detection on the denoised surveillance image data, and perform edge enhancement on the surveillance image data based on the edge detection result; perform brightness normalization on the edge-enhanced surveillance image data to obtain the preprocessed surveillance image data; perform feature extraction on the preprocessed surveillance image data based on a convolutional neural network model, and standardize the extracted feature vectors to obtain feature vectors corresponding to each image.

[0009] By adopting the above technical solution, the image quality is continuously optimized through steps such as denoising, edge enhancement and brightness normalization, and the defect features are magnified. Then, CNN is used to gradually extract image features from low-level to high-level and convert them into vector representations, thereby providing consistent and stable input data for subsequent models.

[0010] Optionally, the method further includes a step of training the packet quality detection model, wherein the training step includes: Acquire a sample image data set; the sample image data set includes packaging box sample image data and pre-annotated defect category labels; Preprocessing the sample image data set and dividing it into a training set and a test set; Inputting the packaging box sample image data in the training set into a pre-built multi-layer perceptron model to obtain a training label result; Compare the training label result with the defect category label in the training set to obtain a training comparison result; The multi-layer perceptron model is iterated according to the training comparison result and the loss function is calculated until the loss function meets the preset condition, thereby obtaining the trained packet quality detection model.

[0011] By adopting the above technical solution and combining the training and optimization of the multi-layer perceptron model, an efficient packet quality detection model is constructed. This model can achieve high-precision packet defect detection on the actual production line and has good generalization ability. It can accurately predict the packet quality status in unseen data and identify the category and location of packet defects, thus providing reliable technical support for packet quality detection on the production line.

[0012] Optionally, after the step of obtaining the trained packet quality detection model, the step further includes: Inputting the packaging box sample image data in the test set into the trained package quality detection model to obtain a test label result; Comparing the test label results based on the defect category labels in the test set to obtain a test comparison result; According to the test comparison results, the performance index of the trained packet quality detection model is evaluated and iterated until a preset performance index is met or a preset number of iterations is reached, thereby obtaining the trained packet quality detection model.

[0013] By adopting the above technical solution, inputting the test set into the model, comparing the test results with the real labels, evaluating the performance indicators, and iteratively optimizing, this application ensures the robustness and accuracy of the packet quality detection model on unseen data. Ultimately, the trained model has the ability to efficiently and accurately classify packet defects, providing strong technical support for production line packet quality monitoring, and effectively improving the operating efficiency and quality management level of the intelligent production line.

[0014] Optionally, the step of projecting the defect marking information into the field of view of the current operator through the AR glasses includes: Receiving the workstation environment image of the current operator collected by the AR glasses; Extracting and identifying the workstation features of the workstation environment image to obtain workstation coordinate information; Obtaining relative coordinate information of the defect position of the package according to the defect detection report; According to the relative coordinate information of the package defect position and the workstation coordinate information, the projection parameter matrix of the AR glasses is configured; based on the configured projection parameter matrix, the defect mark information is projected into the field of view of the current operator through augmented reality technology.

[0015] By adopting the above technical solutions, combined with AR glasses and computer vision technology, a complete closed loop from real-time acquisition of workstation environment images to accurate projection of defect information is achieved. AR technology is used to intuitively feed back defect marking information to operators, helping them to quickly locate and deal with problems, reducing the time for manual defect search and confirmation, significantly improving the efficiency and accuracy of packaging quality monitoring, and providing important technical support for intelligent production lines.

[0016] Optionally, the step of configuring the projection parameter matrix of the AR glasses according to the relative coordinate information of the package defect position and the workstation coordinate information includes: Initialize a projection parameter matrix based on the workstation coordinate information, construct a projection coordinate system and set the workstation coordinate information as a reference origin; Calculating the defect global coordinates of the package defect position in the projection coordinate system according to the relative coordinate information of the package defect position and the workstation coordinate information; Acquire the current viewing angle parameters of the AR glasses in real time; Calculate the intrinsic parameter matrix and the extrinsic parameter matrix of the AR glasses based on the camera calibration algorithm according to the workstation coordinate information, the defect global coordinates and the current viewing angle parameters; The projection parameter matrix is ​​configured according to the intrinsic parameter matrix and the extrinsic parameter matrix; wherein the projection parameter matrix is ​​used to convert the defect global coordinates into AR glasses display plane coordinates.

[0017] By adopting the above technical solution, the whole process calculation from station coordinate initialization, defect location globalization to dynamic projection is realized. The projection parameter matrix constructed based on the intrinsic parameter matrix and the extrinsic parameter matrix ensures the accurate display of defect marks in AR glasses, and realizes dynamic response capability in combination with real-time viewing angle adjustment. This technical solution can provide operators with intuitive and accurate defect positioning support, greatly improving the efficiency and accuracy of packaging quality inspection.

[0018] In the second aspect, the present application provides a production line packaging quality monitoring system based on AR glasses, which adopts the following technical solution: A production line packaging quality monitoring system based on AR glasses, the quality monitoring system comprising: A monitoring image acquisition module, used for acquiring monitoring image data of the packaging box to be tested on the packaging production line in real time; wherein the monitoring image data includes images of different parts of the packaging box to be tested collected from multiple angles; A data processing module, used for preprocessing and feature extraction of the monitoring image data to obtain a feature vector corresponding to each image; The packet quality detection module is used to input the feature vector corresponding to each image into a pre-built packet quality detection model for detection, and obtain the packet quality detection result corresponding to each image; A score calculation module, used for calculating the package integrity score of the package box to be tested based on the package quality detection result corresponding to each image based on a weighted fusion algorithm; a judgment module, used to judge whether the package integrity score is lower than a preset threshold, and if so, output a defect judgment result; a defect detection report generation module, used to generate a defect detection report of the package box to be tested in response to the defect judgment result; A sending module, used for sending the defect detection report to the AR glasses worn by the current operator; A defect marking module, used for marking the package defect position and the package defect category of the package box to be tested according to the defect detection report to obtain defect marking information; A projection module is used to project the defect marking information into the field of view of the current operator through the AR glasses.

[0019] Optionally, the data processing module includes: An image denoising unit, used for performing image denoising on the monitoring image data; An edge detection unit is used to perform edge detection on the denoised monitoring image data, and to perform edge enhancement on the monitoring image data based on the edge detection result; A normalization processing unit, used for performing brightness normalization processing on the edge-enhanced monitoring image data to obtain the pre-processed monitoring image data; The feature extraction unit is used to extract features from the preprocessed monitoring image data based on a convolutional neural network model, and to standardize the extracted feature vectors to obtain feature vectors corresponding to each image.

[0020] In a third aspect, the present application provides a computer device, which adopts the following technical solution: A computer device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the methods in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a first flow chart of a method for monitoring packaging quality of a production line based on AR glasses according to one of the embodiments of the present application.

[0023] Figure 2 This is a second flow chart of a production line packaging quality monitoring method based on AR glasses according to one of the embodiments of the present application.

[0024] Figure 3 This is a third flow chart of a method for monitoring packaging quality of a production line based on AR glasses, one of the embodiments of the present application.

[0025] Figure 4 This is a fourth flow chart of a production line packaging quality monitoring method based on AR glasses, one of the embodiments of the present application.

[0026] Figure 5 This is the fifth flow chart of a production line packaging quality monitoring method based on AR glasses, which is one of the embodiments of the present application.

[0027] Figure 6 This is the sixth flow chart of a production line packaging quality monitoring method based on AR glasses, one of the embodiments of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-6 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0029] An embodiment of the present application discloses a production line packaging quality monitoring method based on AR glasses.

[0030] Reference Figure 1 , a production line packaging quality monitoring method based on AR glasses, the quality monitoring method includes: Step S101, acquiring monitoring image data of a packaging box to be tested on a packaging production line in real time; The monitoring image data includes images of different parts of the package box to be tested collected from multiple angles; In some embodiments, multiple cameras (such as top cameras, side cameras, and front cameras) installed at different positions on the packaging production line are used to capture multi-view images of the packaging box to be tested passing through the production line. These cameras can simultaneously capture images of different parts of the packaging box, such as the top seal, the side edge seal, and the bonding state of the front of the packaging box. The collected image data will be transmitted to the industrial computer through the industrial network to ensure that the image data can be processed in real time without delay.

[0031] Step S102, preprocessing and feature extraction are performed on the monitoring image data to obtain a feature vector corresponding to each image; specifically, the industrial computer preprocesses the collected images (such as denoising, edge enhancement, brightness correction, etc.) to optimize the image quality, and then extracts the deep features of the image through a convolutional neural network (CNN) to generate a feature vector corresponding to each image.

[0032] Step S103, inputting the feature vectors corresponding to each image into a pre-built packet quality detection model for detection, and obtaining the packet quality detection results corresponding to each image; Among them, the package quality inspection model can realize independent inspection of each image to accurately locate the defect category of a single image (such as cracked seal, loose edge, etc.), laying the foundation for subsequent comprehensive judgment.

[0033] Specifically, each feature vector contains multi-dimensional features of the image (e.g. edge shape, sealing tightness, etc.). The package quality inspection model can calculate the weights of these features layer by layer and finally output a defect classification result. For example, the classification result of image 1 may be loose sealing, and the package quality inspection result of image 2 may be uneven edge sealing.

[0034] Step S104, based on a weighted fusion algorithm, a package integrity score of the package box to be tested is calculated according to the package quality detection result corresponding to each image; Among them, the package quality detection results of multiple images are weighted and fused (for example, different weights are assigned to each image) and the comprehensive score S is calculated. If the score is lower than the threshold, it means that the package has serious defects. By integrating the detection results from different perspectives, the quality status of the overall packaging box is quantified, and the weights are flexibly adjusted to meet different production needs, which effectively improves the comprehensiveness and accuracy of the detection and avoids the impact of single image misjudgment on the final quality assessment.

[0035] Specifically, the system assigns different weights to the detection results of each image and uses a weighted fusion algorithm to calculate the comprehensive score of the packet.

[0036] In some embodiments, the calculation formula for the comprehensive score is: In the above formula, n is the number of images, w is i is the weight of the i-th image, indicating the importance of the detection result of the image in the comprehensive score, and must satisfy Di is the packet quality detection result of the i-th image, and the value range is [0,1]. 1 indicates that the detection result is defect-free, and 0 indicates that the detection result is a serious defect. The value between 0 and 1 indicates the severity of the defect.

[0037] It is understandable that the detection results of different images may contribute differently to the overall score, so it is necessary to assign weights to each image; for example, the top image may be more important because the top seal directly affects the integrity of the package, so its weight is higher; the side image is second and has a relatively low weight. In addition, the package quality detection results of each image can be generated Di through the AI ​​model or preset scoring standards, indicating the quality of the package. For example, the score corresponding to a slight crack may be 0.7, and the score corresponding to a loose edge seal may be 0.6.

[0038] Step S105, determining whether the packet integrity score is lower than a preset threshold, if so, jumping to step S106; if not, no operation is performed; Step S106, generating a defect detection report of the package box to be tested; Among them, based on the comparison between the packet integrity score and the set threshold, the system will make the following two judgments: if the score is higher than the threshold, the packet is considered intact and the data does not need to be recorded; if the score is lower than the threshold, a defect detection report is generated and the specific defect location and defect category are recorded.

[0039] For example, if the packet integrity score = 0.65 (lower than the threshold), the defect detection report may include: the defect position is the left edge of the packet, and the defect type is edge misalignment.

[0040] Step S107, sending the defect detection report to the AR glasses worn by the current operator; Specifically, the defect detection report is sent to the corresponding operator's AR glasses through the wireless communication module to ensure timely and accurate information transmission. The defect detection report includes the package defect location and package defect category, such as the problem of misalignment of the left edge of the package.

[0041] Step S108, marking the package defect position and package defect type of the package box to be tested according to the defect detection report to obtain defect marking information; Step S109, projecting the defect marking information into the field of view of the current operator through the AR glasses.

[0042] Among them, AR glasses use augmented reality technology to display the specific location and category information of defects on the real packaging box and project it into the operator's field of view. The operator can directly see the mark of the defect location in the actual view.

[0043] For example, a red frame appears in the operator's field of view marking the left edge area of ​​the packaging box to be tested, and a description is displayed as "Edge seal is not aligned, please check".

[0044] In the above implementation, multi-camera image acquisition and AI detection algorithm are combined to achieve comprehensive intelligent detection of package quality, and defect information is fed back to the operator in real time through AR glasses, forming a closed-loop workflow from defect detection to repair guidance, so that package quality problems on the production line can be discovered and intuitively presented in real time, which not only improves the accuracy and efficiency of detection, but also significantly reduces the errors that may be caused by human intervention, thereby comprehensively optimizing the production quality and management level of the packaging production line, and providing innovative solutions for intelligent industrial production lines.

[0045] Reference Figure 2 As an implementation method of step S102, the steps of preprocessing and feature extraction of monitoring image data to obtain a feature vector corresponding to each image include: Step S201, performing image denoising on monitoring image data; Among them, there may be noise interference (such as dust, light changes) in the industrial production environment, which affects the image quality. Through the denoising algorithm, the random noise in the image can be eliminated to improve the accuracy of subsequent feature extraction. In some embodiments, a Gaussian filter or a median filter can be applied to smooth the background of the image and remove random noise.

[0046] Step S202, performing edge detection on the denoised monitoring image data, and performing edge enhancement on the monitoring image data based on the edge detection result; Among them, packaging defects (such as cracks and loose edge sealing) are often reflected as edge features in the image. By enhancing the edge information of the image, these features can be amplified and the effect of subsequent feature extraction can be improved.

[0047] Specifically, the Sobel operator or the Canny edge detection algorithm may be used to identify the intensity change area in the image, and based on the edge detection result, the detected area is subjected to contrast enhancement processing to make the edge more obvious.

[0048] Step S203, performing brightness normalization processing on the edge-enhanced monitoring image data to obtain pre-processed monitoring image data; Among them, different cameras may cause inconsistent image brightness due to different lighting conditions. This difference will interfere with deep feature extraction, and brightness normalization is required to eliminate the influence of light.

[0049] Specifically, the brightness distribution (such as histogram) of each image is calculated, and the brightness distribution is adjusted using the histogram equalization method to make the image brightness consistent, and the adjusted brightness levels are compared to ensure that the brightness characteristics between different images are consistent.

[0050] Step S204: extract features from the preprocessed monitoring image data based on the convolutional neural network model, and standardize the extracted feature vectors to obtain feature vectors corresponding to each image.

[0051] Among them, the preprocessed image is input into the convolutional neural network (CNN) model to extract its deep feature vectors, which can efficiently capture the edge shape, texture distribution and defect characteristics in the image.

[0052] Specifically, the monitoring image data is input into the first convolution layer of CNN to extract low-level features (such as edges and lines). Through subsequent multi-layer convolution and pooling operations, high-level features (such as complex geometric shapes and textures) are extracted. The final deep features are compressed into a feature vector of a specific dimension through a fully connected layer. The feature vector is output as the feature representation of the packaging box image for subsequent package quality detection.

[0053] It should be noted that the feature vectors extracted by CNN may have inconsistent numerical ranges, so standardization can be performed to improve the stability and efficiency of subsequent models.

[0054] In the above implementation, the image quality is continuously optimized through steps such as denoising, edge enhancement and brightness normalization, and the defect features are amplified. Then, CNN is used to gradually extract image features from low-level to high-level and convert them into vector representations, thereby providing consistent and stable input data for subsequent models.

[0055] Reference Figure 3 As an implementation method of the packet quality detection model, the training steps of the packet quality detection model include: Step S301, obtaining a sample image data set; the sample image data set includes packaging box sample image data and pre-annotated defect category labels; Among them, the sample image dataset is the basis for training the package quality detection model. It must contain a sufficient number of packaging box images and corresponding defect category labels. By controlling the diversity and quality of the sample data, the generalization ability and detection accuracy of the model can be effectively improved.

[0056] In some embodiments, image data of packaging boxes can be collected from an actual packaging production line to ensure that multiple defect types (such as edge cracks, loose seals, etc.) and non-defective samples are covered. In addition, the collected images can be manually or semi-automatically annotated to record the defect category label of each image. For example, image A is annotated as "edge cracks" and image B is annotated as "non-defective".

[0057] Step S302, preprocessing the sample image data set and dividing it into a training set and a test set; Among them, the preprocessing of sample data includes denoising, size normalization, brightness adjustment, etc., which can enhance the training effect by optimizing image quality and ensuring data consistency; the sample data is divided into training set and test set according to a certain ratio (such as 80:20) for model training and performance evaluation to ensure that the model has good generalization ability.

[0058] Step S303, inputting the packaging box sample image data in the training set into a pre-built multi-layer perceptron model to obtain a training label result; Among them, the multi-layer perceptron (MLP) is a commonly used classification model. During the training process, the features of each training set image are input into the model, and the model can output the predicted defect category (training label result). For example, after the feature vector of image A is input into the model, the predicted output is "edge crack"; after the feature vector of image B is input into the model, the predicted output is "no defect".

[0059] In one of the embodiments of the present application, the structure of the packet quality detection model based on the multi-layer perceptron (MLP) mainly includes an input layer, a hidden layer and an output layer. Among them, the input layer receives the image feature vector extracted by the convolutional neural network (CNN), and the dimension of the feature vector is, for example, 128 dimensions or 256 dimensions, depending on the output of the feature extraction. The hidden layer is composed of several fully connected layers, each layer uses ReLU (linear rectification function) as an activation function for nonlinear feature mapping, and the Dropout layer can be used for regularization to prevent overfitting. Finally, the output layer is a fully connected layer, using the Sigmoid activation function to output a probability value (in the range of [0,1]), indicating the probability that the packet is complete. This model structure can effectively extract the classification information in the input features through multi-layer feature mapping and dimensionality reduction, and realizes the binary classification prediction of packet integrity and defects.

[0060] Step S304, comparing the training label result with the defect category label in the training set to obtain a training comparison result; Among them, by comparing the model's prediction results (training label results) with the pre-labeled true labels (defect category labels), the model's classification accuracy and error are calculated as the basis for optimizing the model parameters, thereby clarifying the model's current performance level and providing a clear direction for optimizing the model.

[0061] Step S305, iterating the multi-layer perceptron model according to the training comparison result and calculating the loss function until the loss function meets the preset condition, thereby obtaining a trained packet quality detection model.

[0062] Specifically, in the task of packet quality detection, the goal of model optimization is to minimize the classification error by continuously adjusting the model parameters, thereby improving the model's prediction accuracy of packet integrity (complete or defective). Classification tasks usually use the loss function in deep learning as an optimization indicator, and update the model parameters through optimization algorithms such as gradient descent.

[0063] In one of the embodiments of the present application, the loss function may adopt the cross entropy loss commonly used in binary classification tasks to quantify the difference between the model prediction value and the true label. A smaller loss value indicates that the model classification result is closer to the actual situation.

[0064] Among them, the specific formula of the loss function is: In the above formula, L is the loss function value, which represents the overall error of the current model classification; m is the total number of samples; y i : The true label of the ith sample; y^i: The predicted probability of the ith sample.

[0065] In the above implementation, combined with the training and optimization of the multi-layer perceptron model, an efficient packet quality detection model is constructed. The model can achieve high-precision packet defect detection on the actual production line and has good generalization ability. It can accurately predict the packet quality status in unseen data and identify the category and location of packet defects, thereby providing reliable technical support for packet quality detection on the production line.

[0066] Reference Figure 4 As a further implementation of the monitoring method, after obtaining the trained packet quality detection model in step S305, the method further includes: Step S401, inputting the packaging box sample image data in the test set into the trained package quality detection model to obtain a test label result; After the model training is completed, the packaging box sample image data in the test set is input into the model for inference. The test set is used to simulate the actual effect of unseen data and verify the classification ability of the model.

[0067] Specifically, image feature vectors are extracted from the test set (maintaining the same preprocessing process and feature extraction steps as those during training), and these feature vectors are input into the trained packet quality detection model one by one to obtain the test label results predicted by the model.

[0068] Step S402, comparing the test label results based on the defect category labels in the test set to obtain a test comparison result; Among them, the test label results generated by the model are compared one by one with the defect category labels actually marked in the test set, and the classification accuracy and error indicators are calculated to clarify the specific performance of the model.

[0069] Specifically, the prediction results of each test sample are compared with the true label, the number of correctly classified samples and the number of incorrectly classified samples are counted, and performance indicators such as classification accuracy, recall rate, and F1 value are calculated. Through comparative analysis, the adaptability of the model to the test data is clarified, and potential weaknesses of the model are discovered (such as more classification errors in certain categories). Step S403, based on the test comparison result, the performance index of the trained packet quality detection model is evaluated and iterated until a preset performance index is met or a preset number of iterations is reached, thereby obtaining a trained packet quality detection model.

[0070] Among them, based on the test comparison results, the overall performance of the model is evaluated, including classification accuracy, recall rate, precision and F1 value, and the analysis results of performance indicators are used to determine whether the model meets the preset standards or needs further optimization.

[0071] It should be noted that if the model performance does not meet the preset standard, it is necessary to re-iterate and optimize the model, and gradually improve the model performance by adjusting the training parameters, data set or network structure. Specifically, if the accuracy is low, it may be necessary to increase the training samples or adjust the model structure; if the recall rate is low, it may be necessary to increase the weight of rare category samples. Adjust the training process according to the problem, re-iterate and optimize the model, and repeat the above test process until the model performance reaches the target or the training reaches the maximum number of iterations, so that the final performance of the model meets the preset standard.

[0072] In the above implementation, the test set is input into the model, the test results are compared with the real labels, the performance indicators are evaluated, and iterative optimization is performed. This application ensures the robustness and accuracy of the packet quality detection model on unseen data. Ultimately, the trained model has the ability to efficiently and accurately classify packet defects, providing strong technical support for production line packet quality monitoring, and effectively improving the operating efficiency and quality management level of the intelligent production line.

[0073] Reference Figure 5As an implementation of step S109, the step of projecting the defect marking information into the field of view of the current operator through the AR glasses includes: Step S501, receiving the workstation environment image of the current operator collected by the AR glasses; The built-in camera of the AR glasses currently worn by the operator collects the surrounding environment images in real time, which is mainly used to identify the operator's workstation and provide positioning information for subsequent projection operations. This process relies on the data transmission capability of the camera of the AR glasses and the industrial control system to ensure the real-time and accuracy of the image data.

[0074] Specifically, the camera built into the AR glasses captures the environmental image of the current operator's workstation in real time, including visual features such as the workbench, tools, and equipment, and then transmits it to the industrial computer through the wireless communication module for subsequent processing and analysis.

[0075] Step S502, extracting and identifying the workstation features of the workstation environment image to obtain workstation coordinate information; Among them, by extracting features from the workstation environment image and identifying the iconic features of the workstation (such as equipment layout, workstation number or operation identification, etc.), the specific position and coordinates of the current operator's workstation can be determined, providing basic data for the precise alignment of subsequent projection results.

[0076] Specifically, after receiving the environmental image transmitted by the AR glasses, the industrial computer uses computer vision algorithms (such as SIFT, ORB and other feature extraction algorithms) to extract significant feature points in the workstation image (such as workstation numbers, markers, equipment layouts, etc.), and compares the extracted feature points with the pre-stored workstation template database to identify the current operator's workstation and obtain the workstation coordinate information (such as the three-dimensional coordinates of the workstation, viewing angle, etc.).

[0077] Exemplarily, the image captured by the AR glasses contains the label "Inspection Station 1" next to the packaging equipment. Through the feature extraction algorithm, the system identifies "Inspection Station 1" and obtains the workstation coordinate information (such as coordinates (x1, y1, z1)) to determine the operator's current workstation.

[0078] Step S503, obtaining relative coordinate information of the package defect position according to the defect detection report; The relative coordinates of the package defect position are described in the local coordinate system of the package box, which is usually provided in the defect inspection report, that is, the relative coordinates of the defect in the local coordinate system of the package box (for example, the specific point on the package box (x d ,y d , z d ); Step S504, configuring the projection parameter matrix of the AR glasses according to the relative coordinate information of the package defect position and the workstation coordinate information; Among them, by combining the workstation coordinates (global reference) and the relative coordinates of the defect, the defect position can be mapped to the coordinate system of the AR glasses projection system, and the projection parameter matrix of the AR glasses can be calculated and configured. The calculation of the projection parameter matrix takes into account the global spatial position, viewing angle, focal length and other factors of the defect to ensure that the subsequent projected mark position can be accurately aligned with the defect area of ​​the actual packaging box.

[0079] Step S505 , based on the configured projection parameter matrix, the defect marking information is projected into the field of view of the current operator through augmented reality technology.

[0080] Among them, based on the calculated projection parameter matrix, AR glasses convert the package defect position from global coordinates to the display plane coordinates of AR glasses, and then mark the defect on the display plane of AR glasses and project it into the operator's field of view. Through augmented reality technology, virtual marks (such as defect location marks, defect category descriptions, etc.) are accurately superimposed on the actual operation object to provide real-time feedback.

[0081] Specifically, AR glasses use the configured projection parameter matrix, combined with the operator's perspective and workstation position, to display the defect mark on the real packaging box. The projection result will be updated in real time to ensure that the mark is always aligned with the defect of the packaging box and has high visibility.

[0082] In the above implementation, AR glasses and computer vision technology are combined to achieve a complete closed loop from real-time acquisition of workstation environment images to accurate projection of defect information. AR technology is used to intuitively feed back defect marking information to operators, helping them to quickly locate and deal with problems, reducing the time for manual defect search and confirmation, significantly improving the efficiency and accuracy of packaging quality monitoring, and providing important technical support for intelligent production lines.

[0083] Reference Figure 6 As an implementation of step S504, the step of configuring the projection parameter matrix of the AR glasses according to the relative coordinate information of the package defect position and the workstation coordinate information includes: Step S601, initializing a projection parameter matrix based on the workstation coordinate information, constructing a projection coordinate system and setting the workstation coordinate information as a reference origin; Among them, the workstation coordinate information is used as the reference origin of the projection coordinate system to define the spatial reference of the entire package defect projection. The first step to initialize the projection parameter matrix is ​​to construct the projection coordinate system so that the workstation coordinate information becomes the global spatial reference frame of the calculation.

[0084] Step S602, calculating the defect global coordinates of the package defect position in the projection coordinate system according to the relative coordinate information of the package defect position and the workstation coordinate information; Among them, the location of the package defect is usually represented by the relative coordinates provided by the detection equipment. By mapping the relative coordinates into a projection coordinate system based on the workstation coordinates, the global location of the defect can be obtained for further projection calculation.

[0085] For example, it is assumed that the relative coordinates of the package defect provided by the detection device are (x d ,y d ,z d ), the three-dimensional space coordinate information of the current workstation is (x1, y1, z1), then the global coordinate position (x, y, z) of the defect in the global coordinate system is calculated as: x = x1 + x d ,y=y1+y d ,z=z1+z d .

[0086] Step S603, obtaining the current viewing angle parameters of the AR glasses in real time; Among them, when the operator wears AR glasses, his viewing angle and focal length may change at any time, so it is necessary to obtain the current viewing angle parameters of the AR glasses in real time, including the horizontal viewing angle, vertical viewing angle and focal length, in order to dynamically adjust the projection matrix.

[0087] Specifically, the inertial measurement unit (IMU) of the AR glasses can be used to detect the wearer's head angle changes in real time to obtain the current horizontal and vertical viewing angle parameters (θ h ,θ v ), read the focal length parameter f and optical center offset (c x ,c y ), and use these parameters as input for dynamic projection adjustment to ensure that the projection system can adapt to the wearer's dynamic viewing angle changes and ensure the real-time accuracy of the marking.

[0088] Step S604, calculating the intrinsic parameter matrix and extrinsic parameter matrix of the AR glasses based on the camera calibration algorithm according to the workstation coordinate information, the defect global coordinates and the current viewing angle parameters; Among them, camera calibration is the core of projection accuracy. The intrinsic parameter matrix K describes the optical characteristics of the camera, which can be calculated based on the focal length f and the optical center offset (c x ,c y ) is obtained; the external parameter matrix [R|T] describes the spatial posture of the camera (such as rotation and translation), which can be combined with the current horizontal and vertical viewing angle parameters (θ h ,θ v ) to obtain the rotation matrix R of the camera, and combine it with the workstation coordinate information to obtain the translation vector T.

[0089] Step S605, configuring a projection parameter matrix according to the intrinsic parameter matrix and the extrinsic parameter matrix; wherein the projection parameter matrix is ​​used to convert the defect global coordinates into the AR glasses display plane coordinates.

[0090] Specifically, the projection matrix M is constructed based on the intrinsic parameter matrix and the extrinsic parameter matrix: M = K · [R | T]; Among them, through the precise calculation of the intrinsic parameter matrix and the extrinsic parameter matrix, it is ensured that the construction of the projection parameter matrix can correctly reflect the optical characteristics and spatial posture of the camera.

[0091] Furthermore, the projection parameter matrix is ​​used to convert the defect global coordinates K into the display plane coordinates (X glasses ,y glasses ), using augmented reality technology, the virtual markers are accurately projected into the operator's real field of view, so that the virtual information overlaps with the physical defect location to achieve the purpose of intuitive feedback.

[0092] Specifically, the calculation formula is: (X glasses ,y glasses )=f projection (K, P); In the above formula, K is the global coordinate of the defect, P is the current viewing angle parameter of the AR glasses, and f projection Coordinate transformation function for augmented reality technology.

[0093] In the above implementation, the whole process calculation from station coordinate initialization, defect location globalization to dynamic projection is realized. The projection parameter matrix constructed based on the intrinsic parameter matrix and the extrinsic parameter matrix ensures the accurate display of defect marks in AR glasses, and realizes dynamic response capability in combination with real-time viewing angle adjustment. This technical solution can provide operators with intuitive and accurate defect location support, greatly improving the efficiency and accuracy of packaging quality inspection.

[0094] An embodiment of the present application also discloses a production line packaging quality monitoring system based on AR glasses.

[0095] A production line packaging quality monitoring system based on AR glasses, the quality monitoring system includes: A monitoring image acquisition module is used to acquire monitoring image data of the packaging box to be tested on the packaging production line in real time; wherein the monitoring image data includes images of different parts of the packaging box to be tested collected from multiple angles; The data processing module is used to pre-process and extract features of the monitoring image data to obtain the feature vector corresponding to each image; the packet quality detection module is used to input the feature vector corresponding to each image into the pre-built packet quality detection model for detection to obtain the packet quality detection result corresponding to each image; A score calculation module, used for calculating the package integrity score of the package box to be tested based on the package quality detection result corresponding to each image based on a weighted fusion algorithm; A judgment module is used to judge whether the packet integrity score is lower than a preset threshold, and if so, output a defect judgment result; A defect detection report generating module, used for generating a defect detection report of the package box to be tested in response to the defect judgment result; The sending module is used to send the defect detection report to the AR glasses worn by the current operator; A defect marking module is used to mark the package defect position and package defect category of the package box to be tested according to the defect detection report to obtain defect marking information; The projection module is used to project the defect marking information into the field of view of the current operator through AR glasses.

[0096] In the above implementation, a complete automated quality monitoring process from image acquisition, data processing, defect detection to information projection is realized, which greatly improves the efficiency and accuracy of packaging quality inspection. Through the monitoring image acquisition module, the system can collect images of the packaging box to be tested from multiple angles in real time, ensuring the comprehensiveness of the inspection data; the data processing module preprocesses and extracts features of the image data to ensure the quality of the input data and extract high-dimensional features, providing accurate data support for subsequent inspections; the package quality inspection module combines the AI ​​algorithm to independently detect the multi-view images of the packaging box, and comprehensively evaluates the package integrity based on the weighted fusion algorithm through the scoring calculation module, reducing the possibility of misjudgment due to a single perspective. The system is also equipped with a judgment module to judge the package integrity score. When a defect is found, the defect detection report generation module quickly generates a detailed defect report, including the defect location and type, and the sending module sends the report directly to the AR glasses worn by the operator, realizing real-time transmission and visualization of information. The defect marking module and the projection module use augmented reality technology to intuitively superimpose the defect location and category in the operator's field of view, helping the operator to quickly locate and repair the problem.

[0097] The technical solution of this application solves the problems of low efficiency, incomplete coverage and delayed information transmission of traditional manual inspection. It not only improves the intelligence level of packaging quality inspection, but also provides an efficient and accurate quality control solution for intelligent production lines, which has important practical significance.

[0098] As an implementation of the data processing module, the data processing module includes: An image denoising unit, used for performing image denoising on monitoring image data; An edge detection unit is used to perform edge detection on the denoised monitoring image data, and to perform edge enhancement on the monitoring image data based on the edge detection result; A normalization processing unit, used to perform brightness normalization processing on the edge-enhanced monitoring image data to obtain pre-processed monitoring image data; The feature extraction unit is used to extract features from the preprocessed monitoring image data based on the convolutional neural network model, and to standardize the extracted feature vectors to obtain the feature vectors corresponding to each image.

[0099] A production line packaging quality monitoring system based on AR glasses in an embodiment of the present application can implement any of the above-mentioned production line packaging quality monitoring methods based on AR glasses, and the specific working process of each module in the production line packaging quality monitoring system can refer to the corresponding process in the above-mentioned method embodiment.

[0100] In the several embodiments provided in this application, it should be understood that the provided methods and systems can be implemented in other ways. For example, the system embodiments described above are only illustrative; for example, the division of a certain module is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0101] The embodiment of the present application also discloses a computer device.

[0102] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, a production line packaging quality monitoring method based on AR glasses as described above is implemented.

[0103] The embodiment of the present application also discloses a computer-readable storage medium.

[0104] A computer-readable storage medium stores a computer program that can be loaded by a processor and execute any one of the above-mentioned methods for monitoring packaging quality of a production line based on AR glasses.

[0105] Among them, computer-readable storage media can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus or device; the program code contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0106] It should be noted that in the above embodiments, the description of each embodiment has different emphases, and for the parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0107] The above are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Any feature disclosed in this specification (including the abstract and drawings), unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes. That is, unless otherwise stated, each feature is only an example of a series of equivalent or similar features.

Claims

1. A production line packaging quality monitoring method based on AR glasses, characterized in that: The quality control method comprises: Acquire monitoring image data of the packaging box to be tested on the packaging production line in real time; wherein the monitoring image data includes images of different parts of the packaging box to be tested collected from multiple angles; Preprocessing and feature extraction are performed on the monitoring image data to obtain a feature vector corresponding to each image; Input the feature vectors corresponding to each image into the pre-built packet quality detection model for detection, and obtain the packet quality detection results corresponding to each image; Based on a weighted fusion algorithm, a package integrity score of the package box to be tested is calculated according to the package quality detection result corresponding to each image; Determining whether the package integrity score is lower than a preset threshold, and if so, generating a defect detection report for the package box to be tested; Sending the defect detection report to the AR glasses worn by the current operator; Marking the package defect position and package defect category of the package box to be tested according to the defect detection report to obtain defect marking information; The defect marking information is projected into the field of view of the current operator through the AR glasses.

2. The method for monitoring packaging quality of a production line based on AR glasses according to claim 1, characterized in that: The steps of preprocessing and feature extracting the monitoring image data to obtain a feature vector corresponding to each image include: Performing image denoising on the monitoring image data; Perform edge detection on the denoised surveillance image data, and perform edge enhancement on the surveillance image data based on the edge detection result; Performing brightness normalization processing on the edge-enhanced monitoring image data to obtain the pre-processed monitoring image data; Based on the convolutional neural network model, feature extraction is performed on the preprocessed monitoring image data, and the extracted feature vectors are standardized to obtain the feature vector corresponding to each image.

3. The method for monitoring packaging quality of a production line based on AR glasses according to claim 1, characterized in that: The method further includes a step of training the packet quality detection model, wherein the training step includes: Acquire a sample image data set; the sample image data set includes packaging box sample image data and pre-annotated defect category labels; Preprocessing the sample image data set and dividing it into a training set and a test set; Inputting the packaging box sample image data in the training set into a pre-built multi-layer perceptron model to obtain a training label result; Compare the training label result with the defect category label in the training set to obtain a training comparison result; The multi-layer perceptron model is iterated according to the training comparison result and the loss function is calculated until the loss function meets the preset condition, thereby obtaining the trained packet quality detection model.

4. The method for monitoring packaging quality of a production line based on AR glasses according to claim 3, characterized in that: After the step of obtaining the trained packet quality detection model, the step further includes: Inputting the packaging box sample image data in the test set into the trained package quality detection model to obtain a test label result; Comparing the test label results based on the defect category labels in the test set to obtain a test comparison result; According to the test comparison results, the performance index of the trained packet quality detection model is evaluated and iterated until a preset performance index is met or a preset number of iterations is reached, thereby obtaining the trained packet quality detection model.

5. A production line packaging quality monitoring method based on AR glasses according to any one of claims 1 to 4, characterized in that: The step of projecting the defect marking information into the field of view of the current operator through the AR glasses includes: Receiving the workstation environment image of the current operator collected by the AR glasses; Extracting and identifying the workstation features of the workstation environment image to obtain workstation coordinate information; Obtaining relative coordinate information of the defect position of the package according to the defect detection report; According to the relative coordinate information of the package defect position and the workstation coordinate information, configuring the projection parameter matrix of the AR glasses; Based on the configured projection parameter matrix, the defect marking information is projected into the field of view of the current operator through augmented reality technology.

6. The method for monitoring packaging quality of a production line based on AR glasses according to claim 5, characterized in that: According to the relative coordinate information of the package defect position and the workstation coordinate information, the step of configuring the projection parameter matrix of the AR glasses includes: Initialize a projection parameter matrix based on the workstation coordinate information, construct a projection coordinate system and set the workstation coordinate information as a reference origin; Calculating the defect global coordinates of the package defect position in the projection coordinate system according to the relative coordinate information of the package defect position and the workstation coordinate information; Acquire the current viewing angle parameters of the AR glasses in real time; Calculate the intrinsic parameter matrix and the extrinsic parameter matrix of the AR glasses based on the camera calibration algorithm according to the workstation coordinate information, the defect global coordinates and the current viewing angle parameters; The projection parameter matrix is ​​configured according to the intrinsic parameter matrix and the extrinsic parameter matrix; wherein the projection parameter matrix is ​​used to convert the defect global coordinates into AR glasses display plane coordinates.

7. A production line packaging quality monitoring system based on AR glasses, characterized in that: The quality monitoring system comprises: A monitoring image acquisition module, used for acquiring monitoring image data of the packaging box to be tested on the packaging production line in real time; wherein the monitoring image data includes images of different parts of the packaging box to be tested collected from multiple angles; A data processing module, used for preprocessing and feature extraction of the monitoring image data to obtain a feature vector corresponding to each image; The packet quality detection module is used to input the feature vector corresponding to each image into a pre-built packet quality detection model for detection, and obtain the packet quality detection result corresponding to each image; A score calculation module, used for calculating the package integrity score of the package box to be tested based on the package quality detection result corresponding to each image based on a weighted fusion algorithm; A judgment module, used to judge whether the packet integrity score is lower than a preset threshold, and if so, output a defect judgment result; A defect detection report generating module, used for generating a defect detection report of the packaging box to be tested in response to the defect judgment result; A sending module, used for sending the defect detection report to the AR glasses worn by the current operator; A defect marking module, used for marking the package defect position and the package defect category of the package box to be tested according to the defect detection report to obtain defect marking information; A projection module is used to project the defect marking information into the field of view of the current operator through the AR glasses.

8. The production line packaging quality monitoring system based on AR glasses according to claim 7 is characterized in that: The data processing module comprises: An image denoising unit, used for performing image denoising on the monitoring image data; An edge detection unit is used to perform edge detection on the denoised monitoring image data, and to perform edge enhancement on the monitoring image data based on the edge detection result; A normalization processing unit, used for performing brightness normalization processing on the edge-enhanced monitoring image data to obtain the pre-processed monitoring image data; The feature extraction unit is used to extract features from the preprocessed monitoring image data based on a convolutional neural network model, and to standardize the extracted feature vectors to obtain feature vectors corresponding to each image.

9. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 6.

Images