Intelligent toothpaste tube tail date code abnormity detection system

Abstract

The invention discloses an intelligent toothpaste tube tail date code anomaly detection system, which comprises an image acquisition and image labeling module for collecting toothpaste tube pictures and carrying out picture positioning labeling and character recognition labeling; the model positioning and model recognition module is used for training a character positioning model and a character recognition model respectively to obtain a reasoning file; the model reasoning file deployment module is used for deploying a reasoning file converted after model training by utilizing a tensorrt reasoning framework; the alarming and rejecting module is used for acquiring the batch number issued by the PLC, acquiring a toothpaste tube tail date code picture in real time and carrying out identification and comparison; when it is detected that the code spraying characters at the tail of the toothpaste tube are inconsistent with the batch number issued by the PLC, an NG signal is transmitted to a PLC system of a production line, and the removing action is started. According to the invention, abnormally printed products can be rejected in real time, so that manpower-related inspection and management work is reduced, the supervision cost of an enterprise is reduced, and the production quality of the products is improved.

Description

Technical Field

[0001] The present invention relates to a product code anomaly detection system, in particular to an intelligent toothpaste tube end date code anomaly detection system. Background Art

[0002] Toothpaste has become an indispensable product in our daily lives, and the production process is complex and sophisticated. First, the toothpaste is made through operations such as preparing small ingredients, preparing purified water, and transferring powder and liquid materials. After the paste is made, the toothpaste is transported through a filling machine to fill the tubes. After the tubes are filled, they are sealed, the date code is printed, and the toothpaste in the tubes is packed into cartons. Finally, the finished product is stored.

[0003] The toothpaste production, tube filling, date code printing, and packaging and warehousing processes are all highly automated. Therefore, during this automated production process, there is always a certain probability that the end of the toothpaste tube will be wrinkled or the date code will not be clearly printed, resulting in some low-quality products in this process. These products will have a negative impact on consumers when they enter the market, thus affecting the overall product reputation and ultimately leading to a decline in sales. Currently, the date code on the end of the toothpaste tube is inspected manually by sampling, which is used to estimate the overall quality of the date code production. Although it can represent the overall quality of the date code printing to a certain extent, this traditional manual sampling test also has the following problems: 1. Insufficient intelligence. Manual sampling inspection relies entirely on manual inspection and observation, which does not meet the requirements and standards of Industry 4.0 intelligence and the promotion and application of AI intelligence in the industrial market, greatly reducing production efficiency and quality.

[0004] 2. Many missed inspections. During the automated production process, tens of thousands of toothpastes may be produced every day. Manual testing is impossible to test all of them. A small number of samples must be taken for testing. This will result in a large amount of toothpaste not being tested, which may lead to poor quality products entering the market. Therefore, manual testing often misses inspections and cannot meet high-quality production requirements.

[0005] 3. High labor costs. Different production lines need to be assigned a staff member to conduct inspections, which will result in high labor costs and a large amount of cost consumption.

[0006] In recent years, with the continuous development and popularization of artificial intelligence, the integration of AI intelligent detection and industry has become increasingly close. Simultaneously, with the continuous improvement of computer computing power and the continuous development of electronic equipment, communication equipment, and hardware, the integration of vision and industry has been promoted, leading to the increasingly widespread application of AI algorithms in industry. Therefore, it is necessary to provide an efficient, flexible, and real-time intelligent system for detecting anomalies in the date code on the end of toothpaste tubes. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide an intelligent system for detecting abnormal date codes on the end of toothpaste tubes, which can eliminate products with abnormal printing in real time, thereby reducing manpower-related inspection and management work, lowering the company's supervision costs, and improving product production quality.

[0008] The technical solution adopted by the present invention to solve the above technical problems is to provide an intelligent toothpaste tube end date code anomaly detection system, including an image acquisition and image annotation module: collects a certain number of toothpaste tube end date code images, performs image positioning annotation and character recognition annotation, and obtains annotation files for model training; a model positioning and model recognition module: trains character positioning and character recognition models respectively to obtain inference files; a model inference file deployment module: deploys the inference files converted after model training using the TensorRT inference framework; an alarm and rejection module: obtains the batch number issued by the PLC, turns on the camera to collect the toothpaste tube end date code image in real time, uses the trained model to identify the collected image, and compares the recognition result with the obtained batch number; when it is detected that the coded characters on the toothpaste tube end are inconsistent with the batch number issued by the PLC, the PCL starts the rejection action and triggers the alarm system to sound an alarm.

[0009] Furthermore, the image acquisition and image annotation module uses a camera to capture and annotate images of date codes on the end of toothpaste tubes with different batch numbers, different dates, different brightness, and different backgrounds. The file used for instance segmentation and character positioning is labeled as a Label.txt label file, and the file used for OCR character recognition is labeled as a rec_gt.txt label file.

[0010] Furthermore, the model positioning and model recognition module first uses a deep learning instance segmentation model to locate the two rows of characters on the date code image at the end of the toothpaste tube. After positioning is completed, the two rows of characters are cut out according to the positioning box, and then fed into the deep learning OCR recognition model for character recognition, and finally the character recognition results are obtained.

[0011] Furthermore, after the model positioning and model identification modules are trained, a model file is obtained, and the model file is converted into an onnx file that can be parsed by the tensorrt deployment framework.

[0012] Furthermore, the model inference file deployment module parses the onnx file to generate a trt file, and integrates the generated trt file into the software interface end using C++.

[0013] Furthermore, the model inference file deployment module deploys the deep learning instance segmentation model in C++, intercepts the image of the character recognition framework segmented from the instance, and then uses the tensorrt framework to deploy the deep learning OCR recognition model in C++, and passes the intercepted character images into the deep learning OCR recognition model in sequence for character recognition.

[0014] Furthermore, the model inference file deployment module controls the frame rate of the screen to be between 15 and 25 per second.

[0015] Furthermore, the software interface end integrates a camera SDK, a voice alarm, an IO module and a communication module. The PLC obtains the batch number issued from the database server, and transmits the batch number to the IO module. The software interface end reads the signal of the IO module to start the camera to collect pictures, and then transmits the collected pictures to the model positioning and model recognition module for image recognition, and compares the recognition result with the obtained batch number. If OK, continue the above action. If NG, transmit the signal to the PLC through the IO module, and the PLC drives the rejection system to perform a rejection action to reject the toothpaste that is not printed clearly or has wrinkles, and at the same time drive the voice alarm through the IO module to alarm.

[0016] Furthermore, the installation direction of the camera is parallel to the vertical direction of the toothpaste tube. The image acquisition and image annotation module acquires images through 45-degree inclined mirror reflection in the customized light source, and a light-diffusing plate is installed in front of the customized light source. The model positioning and model recognition module uses perspective transformation to correct the tilt angle of the image, then enhances the corrected image, and finally feeds it to the deep learning OCR recognition model for character recognition.

[0017] Furthermore, the model positioning and model recognition module uses the following perspective transformation formula to scale, rotate and translate the image's tilt angle: Among them, u, v are the left sides of the original image, x', y', w' are the new coordinates after transformation, and the actual coordinates x, y are obtained by This transformation method can achieve the same effect as the image, is the transformation matrix, represents a linear transformation, For translation, Produce perspective transformation, and finally get the transformation formula as follows: .

[0018] Compared with the prior art, the present invention has the following beneficial effects: the intelligent toothpaste tube tail date code anomaly detection system provided by the present invention performs real-time detection through deep learning instance segmentation and OCR character recognition algorithm; the monitoring of date code inkjet printing anomalies is more intelligent and efficient, and is suitable for detection in different scenarios. At the same time, the system performs date code inkjet printing anomaly warning, and combines the PLC rejection function and action of the production line to reject abnormally printed products in real time, thereby reducing manpower-related inspection and management work, reducing the company's supervision costs, and improving product production quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of the deep learning neural network model training of the present invention; Figure 2 This is the overall flow chart of the intelligent toothpaste tube end date code anomaly detection system of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described below with reference to the accompanying drawings and examples.

[0021] The intelligent toothpaste tube end date code anomaly detection system provided by the present invention includes the following modules: 1. Image acquisition and image annotation module The present invention uses a custom light source mirror and a diffuser in front of the custom light source for image capture. The custom light source uses the principle of mirror reflection for image capture. Because the toothpaste tube filling space in the present invention is very narrow, the camera and the toothpaste tube cannot be photographed vertically. There is insufficient space to install the camera at the vertical front end of the tube tail. Therefore, the camera is installed parallel to the vertical direction of the toothpaste tube. Image capture is performed based on the principle of 45-degree inclined mirror reflection in the custom light source. A diffuser is installed in front of the custom light source to prevent overexposure and reflection, thereby maximizing image quality in this scenario. Initially, the camera is used to capture images of date codes from the tube tails of toothpaste tubes of different batches and dates. A certain number of images are collected for image location annotation and character recognition annotation, resulting in an annotated txt file for use in model training.

[0022] 2. Model positioning and model recognition module This model primarily utilizes the deep learning PaddlePaddle framework. Building on Baidu's years of deep learning technology research and business applications, PaddlePaddle integrates a core framework, basic model library, end-to-end development kit, a rich set of tools, and the Galaxy Community. It is China's first independently developed, feature-rich, and open-source industrial-grade deep learning platform. First, a deep learning instance segmentation model is used to locate characters. The toothpaste tube has two rows of characters, and the first and second rows are classified. Therefore, the two rows of characters in the image are located. After location is complete, the two rows of characters are cut out according to the positioning box, and the two rows of characters are cut out one by one. After the toothpaste tube is filled with toothpaste by the filling machine, the rotating and translating mechanism will rotate and translate the toothpaste tube to the position of the pneumatic suction nozzle. The toothpaste tube will be lifted up by the bottom push rod, and the pneumatic suction nozzle will suck the toothpaste tube for boxing. Due to spatial position limitations, the camera takes pictures when the toothpaste tube is lifted up by the push rod. During the lifting process, the toothpaste tube will be tilted at a certain angle due to the imbalance of gravity. When the camera collects the character image at the end of the toothpaste tube, the characters will have a certain tilt angle. Therefore, after the characters are located and extracted, the character image needs to be corrected, and then the corrected image needs to be enhanced. Finally, it is fed to the deep learning OCR (Optical Character Recognition) recognition model for character recognition to obtain the character recognition result.

[0023] This invention primarily uses the principle of perspective transformation to correct images. Perspective transformation is the process of projecting an image from one viewing plane to another, so perspective transformation is also called projection mapping. In an affine transformation of an image, the required transformation matrix is a 2×3 two-dimensional plane transformation matrix. However, perspective transformation is essentially a three-dimensional spatial transformation. Based on the coordinate variance, projecting three-dimensional coordinates to another viewing plane requires a completely different transformation matrix M. The transformation formula is: Among them, u, v are the left sides of the original image, x', y', w' are the new coordinates after transformation, and the actual coordinates x, y can be obtained by This transformation method can achieve the effects of image scaling, rotation, translation, etc. The transformation matrix Can be disassembled into four parts. represents a linear transformation, Indicates translation For translation, Producing perspective transformation, the final transformation formula is as follows: .

[0024] 3. Model inference file deployment module This module mainly deploys the inference files converted after model training, using the TensorRT inference framework for deployment. It mainly deploys the deep learning instance segmentation model in C++, captures images of the character recognition framework generated by the instance segmentation, and then uses the TensorRT framework to deploy the deep learning OCR recognition model in C++. The captured character images are sequentially passed to the deep learning OCR recognition model for character recognition. The total process fps (frames per second) is between 15 and 25, which meets the speed requirements of on-site production.

[0025] 4. Alarm and rejection module When abnormal inkjet printing is detected at the end of the toothpaste tube, the model detects that the characters are inconsistent with the batch number issued by the PLC, the model will report NG, and the software will transmit the NG signal to the PLC system of the production line. The PCL will start the rejection action according to the signal, and the software will trigger the alarm system to sound an alarm.

[0026] Figure 1 This is the training flow chart of the deep learning neural network model of the present invention. First, the camera captures the printed images of the date code on the end of the toothpaste tube. The present invention collects a total of 8,460 images of the date code on the end of the toothpaste tube with different batch numbers, different brightness, and different backgrounds. These more than 8,000 images are annotated to obtain Label.txt and rec_gt.txt label files. The Label.txt label file is used for instance segmentation and character positioning, and the rec_gt.txt label file is used for OCR character recognition. The character positioning and character recognition models are trained respectively to obtain model files, which are then converted into onnx files (Open Neural Network Exchange) that can be parsed by the TensorRT deployment framework.

[0027] The present invention uses the tensorrt framework for acceleration model inference for deployment, uses the tensorrt framework to parse the trained onnx model to generate a trt file, and integrates the generated trt file into the software interface using C++.

[0028] Figure 2This is the overall flow chart of the intelligent toothpaste tube end date code anomaly detection system of the present invention. The software interface of the present invention realizes the joint debugging of the entire production line through integrated algorithms, camera SDK, voice alarm and information communication. The specific process is that the database server sends the job number to the PLC, the PLC transmits the job to the IO module, and the software control interface starts the camera to collect pictures by reading the signal of the IO module, and then transmits the collected pictures to the algorithm module for image recognition. Finally, the recognition result is compared with the original batch number. If it is OK, the above action is continued. If it is NG, the signal is transmitted to the PLC through the IO module, and the PLC drives the rejection system to perform the rejection action to reject the toothpaste with unclear printing or wrinkles. At the same time, the alarm system is driven by the IO module to sound an alarm.

[0029] Finally, in the HMI interface display part, the total number of OK and NG identifications is analyzed and displayed on the front-end page for staff to observe and analyze the test results, preparing for the next model upgrade and software upgrade.

[0030] The intelligent toothpaste tube end date code anomaly detection system provided by the present invention has the following advantages: 1. This invention combines deep learning with hardware to achieve AI intelligent detection to determine whether the date code printed on the end of the toothpaste tube is clear, thus realizing intelligent management; 2. The present invention detects the characters on the end of the toothpaste tube in real time, and detects each toothpaste, achieving a real-time detection effect. This ensures that the date code on the end of each toothpaste tube can be detected, thereby preventing inferior products from entering the market and optimizing product quality from the source. 3. It improves the efficiency of industrial production, frees up manual inspection, and reduces the labor cost of enterprises.

[0031] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. An intelligent toothpaste tube end date code anomaly detection system, characterized in that: include: Image acquisition and image annotation module: This module collects a certain number of images of date codes on the end of toothpaste tubes, performs image location annotation and character recognition annotation, and generates annotation files for model training. Model positioning and model recognition modules: train character positioning and character recognition models respectively to obtain inference files; Model inference file deployment module: deploys the inference files converted after model training using the TensorRT inference framework; Alarm and rejection module: Obtain the batch number issued by the PLC, turn on the camera to capture the date code image at the end of the toothpaste tube in real time, use the trained model to recognize the captured image, and compare the recognition result with the obtained batch number; when it detects that the inkjet code characters at the end of the toothpaste tube are inconsistent with the batch number issued by the PLC, the PCL starts the rejection action and triggers the alarm system to sound an alarm.

2. The intelligent toothpaste tube end date code abnormality detection system according to claim 1, characterized in that: The image acquisition and image annotation module uses a camera to capture and annotate images of date codes on the end of toothpaste tubes with different batch numbers, different dates, different brightness, and different backgrounds. The file used for instance segmentation and character positioning is annotated as a Label.txt label file, and the file used for OCR character recognition is annotated as a rec_gt.txt label file.

3. The intelligent toothpaste tube end date code abnormality detection system according to claim 1, characterized in that: The model positioning and model recognition module first uses a deep learning instance segmentation model to locate the two rows of characters on the date code image at the end of the toothpaste tube. After positioning is completed, the two rows of characters are cut out according to the positioning frame, and then fed into the deep learning OCR recognition model for character recognition, and finally the character recognition results are obtained.

4. The intelligent toothpaste tube end date code abnormality detection system according to claim 1, characterized in that: After training the model positioning and model identification modules, a model file is obtained, and the model file is converted into an onnx file that can be parsed by the tensorrt deployment framework.

5. The intelligent toothpaste tube end date code anomaly detection system according to claim 4, characterized in that: The model reasoning file deployment module parses the onnx file to generate a trt file, and integrates the generated trt file into the software interface end using C++.

6. The intelligent toothpaste tube end date code abnormality detection system according to claim 5, characterized in that: The model inference file deployment module deploys the deep learning instance segmentation model in C++, intercepts the image of the character recognition framework obtained by instance segmentation, and then uses the tensorrt framework to deploy the deep learning OCR recognition model in C++, and sequentially passes the intercepted character images into the deep learning OCR recognition model for character recognition.

7. The intelligent toothpaste tube end date code anomaly detection system according to claim 6, characterized in that: The model inference file deployment module controls the frame rate of the screen to be between 15 and 25 frames per second.

8. The intelligent toothpaste tube end date code anomaly detection system according to claim 5, characterized in that: The software interface integrates a camera SDK, a voice alarm, an IO module and a communication module. The PLC obtains the batch number issued from the database server, and transmits the batch number to the IO module. The software interface reads the signal of the IO module to start the camera to collect pictures, and then transmits the collected pictures to the model positioning and model recognition module for image recognition. The recognition result is compared with the obtained batch number. If OK, the above action is continued. If NG, the signal is transmitted to the PLC through the IO module. The PLC drives the rejection system to perform a rejection action to reject toothpaste that is not printed clearly or has wrinkles. At the same time, the voice alarm is driven by the IO module to alarm.

9. The intelligent toothpaste tube end date code abnormality detection system according to claim 1, characterized in that: The camera is installed in a direction parallel to the vertical direction of the toothpaste tube. The image acquisition and image annotation module acquires images through 45-degree inclined mirror reflection in the customized light source, and a light-diffusing plate is installed in front of the customized light source. The model positioning and model recognition module uses perspective transformation to correct the tilt angle of the image, then enhances the corrected image, and finally feeds it to the deep learning OCR recognition model for character recognition.

10. The intelligent toothpaste tube end date code abnormality detection system according to claim 9, characterized in that: The model positioning and model recognition modules use the following perspective transformation formula to scale, rotate, and translate the image according to its tilt angle: Among them, u, v are the left sides of the original image, x', y', w' are the new coordinates after transformation, and the actual coordinates x, y are obtained by This transformation method can achieve the same effect as the image, is the transformation matrix, represents a linear transformation, For translation, Produce perspective transformation, and finally get the transformation formula as follows: 。

Images