Material blockage detection and state monitoring method based on deep learning

Abstract

The invention discloses a material blockage detection and state monitoring method based on deep learning, and the method comprises the steps of data collection and labeling, data amplification and preprocessing, YOLOv5 model optimization and training, real-time detection and alarm, and the like. The real-time performance and accuracy of detection are greatly improved, the whole process is automated from video acquisition and data processing to model detection and alarm, manual intervention is greatly reduced, and the production efficiency is improved. An efficient and reliable solution is provided for material blocking detection and storage cabinet state monitoring in the cigarette production and manufacturing process, various problems existing in the prior art are solved, and the production management level and the production efficiency are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of cigarette manufacturing, and particularly to a method for detecting material blockage and monitoring the state of storage cabinets and conveyor belts in the cigarette production and manufacturing industry. Background Art

[0002] With the continuous development of the cigarette production and manufacturing industry, the degree of automation and intelligence of production lines has been gradually improved. However, in the process of cigarette production, the problem of material blockage is still one of the key factors affecting production efficiency and product quality. Traditional material blockage detection methods mainly rely on manual inspections and simple sensor detections. These methods are not only time-consuming and laborious, but also prone to missed detections and false alarms, and are difficult to meet the high-efficiency requirements of modern production lines.

[0003] For example, mechanical sensors and alarm systems: This method uses mechanical sensors installed on the production line, such as photoelectric sensors and pressure sensors, to monitor the flow state of materials in real time. When material blockage is detected, the sensor will trigger the alarm system to notify the operator for handling. The advantage of this method is strong real-time performance and fast reaction speed, but the disadvantage is that it is easily affected by mechanical wear and environmental factors, resulting in a decrease in the sensitivity of the sensor and frequent false alarms. Some enterprises have also begun to adopt a monitoring system based on image processing technology. By installing cameras at key positions, real-time images of the production line are collected, and image processing algorithms are used to analyze the flow state of materials and the occupancy of storage cabinets. Once an abnormal situation is detected, the system will automatically send an alarm signal. The advantage of this method is non-contact monitoring and is not easily affected by mechanical wear, but the accuracy and real-time performance requirements of the image processing algorithm are relatively high, feature extraction and preprocessing are more cumbersome, and it needs to be developed and designed for specific scenarios, and the versatility is not high.

[0004] In recent years, with the development of artificial intelligence technology, some enterprises have begun to try to use machine learning algorithms for material blockage and storage cabinet state detection. For example, by collecting a large amount of production line image data, a deep learning model is trained to automatically identify material blockage and abnormal states. The advantage of this method is that it can handle complex production environments and changing working conditions, and the recognition accuracy is high, but training the model requires a large amount of labeled data, and the model deployment and maintenance costs are relatively high.

[0005] It can be seen that although the existing material blockage detection technologies have solved the material blockage problem in production to a certain extent, there are still the following disadvantages and deficiencies: 1. For the monitoring system and manual monitoring method Slow reaction time: The traditional monitoring system relies on manual labor. The monitoring personnel judge the material blockage situation through video images, and the reaction speed depends on the observation ability and reaction speed of people, making it difficult to achieve real-time and efficient monitoring; High labor intensity: Monitoring personnel need to stare at the screen for a long time, resulting in high labor intensity, which is prone to fatigue and affects the accuracy of judgment. High lag: Manual detection has a lag. When the operator discovers the material blockage problem, it may have already had an adverse impact on production.

[0006] 2. For mechanical sensors and alarm systems Prone to mechanical wear and environmental factors: Mechanical sensors such as photoelectric sensors and pressure sensors are prone to mechanical wear and environmental factors (such as dust, temperature) during long-term use, resulting in a decrease in sensor sensitivity and detection accuracy. Frequent false alarms: Sensors are prone to false alarms during detection, especially in complex production environments, and false alarms will increase the difficulty of maintenance and management.

[0007] 3. For image processing monitoring methods High algorithm complexity: Image processing technology relies on complex image processing algorithms and a large number of image feature extraction and preprocessing steps. These algorithms need to be developed and debugged for specific scenarios, and the versatility is not high. High computing resource requirements: Real-time image processing requires high-performance computing resources to support, otherwise it will cause processing delays and affect the real-time performance and accuracy of detection.

[0008] 4. Intelligent monitoring based on machine learning High data collection and annotation costs: The training of machine learning models requires a large amount of annotated data, and the data collection and annotation process is cumbersome and costly.

[0009] High hardware and maintenance costs: Special multi-camera stereo cameras need to be installed and proprietary computing devices need to be configured. The hardware cost is relatively high, and a separate model needs to be established. Summary of the Invention

[0010] In view of the above deficiencies, the present invention provides a method for detecting material blockage and monitoring the state based on deep learning, aiming to solve the problems in the prior art such as slow monitoring response time, high labor intensity of manual monitoring, susceptibility of sensors to environmental influences, and high complexity of image processing algorithms. Through the intelligent material blockage detection and cabinet state monitoring method based on the YOLOv5 model, the present invention can achieve real-time, accurate, and low-cost material blockage detection and alarm, and improve the automation and intelligence level in the cigarette production and manufacturing process.

[0011] The technical solution adopted by the present invention to solve its technical problems is as follows: A method for detecting material blockage and monitoring the state based on deep learning, including S1 Data collection and annotation Obtain the material blockage videos taken by multiple cameras, and intercept the videos into static images at a rate of one second per frame. Manually annotate the intercepted images using the LabelImg tool, marking the material blockage scenarios and the storage and transfer scenarios of normal materials therein to form a preliminary dataset; S2 Data Augmentation and Preprocessing Perform data augmentation on the annotated image dataset; Each image is augmented by 5 times through rotation, flipping, blurring, adding noise, changing brightness, etc. to increase the diversity and robustness of the dataset; Divide the augmented dataset into a training set and a validation set according to a certain proportion; S3 Model Optimization Based on the YOLOv5 model, perform image detection on the blockage situation of tobacco leaves or cut tobacco. To optimize the small target detection ability, the following optimizations and improvements are made to the YOLOv5 model; S3.1 Configure the Anchor Module First, analyze the dataset and count the size range of small targets in the samples of the dataset; Then, in the corresponding configuration file of the YOLOv5 model, configure the Anchor module based on the size range of the small targets, and adjust the size range according to the aspect ratio of the targets; S3.2 Introduce the CBAM Attention Module The CBAM attention module is set in the early stage of the convolutional layer (especially the first layer or the first few layers) and after the upsampling layer to enhance the model's attention to important regions at this stage. Especially when there are small targets or local information is missing, the CBAM attention module helps the model focus on these regions through channel and spatial attention; S3.3 Add the Dice Loss Function The loss function of the YOLOv5 model includes classification loss, regression loss, and Dice Loss; When calculating the loss, the Dice Loss is weighted and combined with the original classification loss and regression loss. For each prediction box, calculate the overlapping area between the prediction box and the ground truth box through the Dice Loss, and adjust the weight according to the needs of model training to avoid affecting the training process of other loss terms; S4 Model Training Use the optimized YOLOv5 model to train on the dataset, and optimize the model by adjusting hyperparameters. Adopt the method of transfer learning until the detection accuracy of the model on the validation set reaches the expected standard; after training is completed, generate the final weight file; S5 Real-time Detection and Alarm Input the real-time video stream of the virtual camera into the YOLOv5 model to detect the blockage situation in the cigarette production line and output the results; When detecting material blockage, based on the set confidence threshold and alarm trigger mechanism, check whether the confidence of the output result when the yolov5 model detects material blockage is higher than 50%. If it is higher than 50%, an alarm will be triggered through a pop-up window on the GUI interface, and the alarm information will be recorded for the operator to handle in a timely manner.

[0012] As an improvement of the above technical solution, S1 data collection and annotation includes S11 Acquisition and processing of multiple video streams Use multiple cameras installed on the top of the production site to collect material blockage video streams through the ONVIF protocol, and combine multiple video streams into one video stream through a video decoder; S12 Virtual camera simulation Input the HDMI signal output by the video decoder into the PC through an HDMI video capture card and simulate it as the output of a virtual camera for detection by the YOLOv5 model.

[0013] As an improvement of the above technical solution, the camera is a plurality of rotating monitoring cameras installed in the underground leaf storage room of the cigarette making process to cover the storage cabinet, conveyor belt, and discharge port areas, and the camera is connected to the video decoder through the ONVIF protocol.

[0014] As an improvement of the above technical solution, the training parameters of the YOLOv5 model are: batch size = 16, epochs = 100, learning rate = 0.001; After training is completed, a final weight file is generated, and the accuracy of the validation set reaches more than 95%.

[0015] The application of this material blockage detection and status monitoring method based on deep learning includes Collect material blockage video streams through multiple cameras, combine multiple video streams into one video stream through a video decoder, and transmit it to the LED controller through the HDMI channel for display on the output screen; Input the HDMI signal output by the video decoder into the PC through an HDMI video capture card and simulate it as the output of a virtual camera; Input the real-time video stream of the virtual camera into the YOLOv5 model to detect the material blockage situation and the status of the storage cabinet in the cigarette production line; Output the detection results in real time and display them.

[0016] The beneficial effects brought by the present invention are: 1. Improve the accuracy and real-time performance of material blockage detection: The present invention conducts real-time analysis on video images through the YOLOv5 deep learning model, and can accurately identify material blockage situations and the status of storage cabinets. Compared with traditional image recognition algorithms and sensors, the detection accuracy is greatly improved, and the false alarm rate is significantly reduced. The ability of real-time analysis and rapid response enables the monitoring system to send alarm signals at the initial stage of material blockage, effectively reducing the production downtime caused by material blockage.

[0017] 2. Reduce the labor intensity of manual monitoring: This method uses deep learning-based image recognition, reducing the dependence on manual monitoring. Monitoring personnel no longer need to stare at the screen for a long time and only intervene when the system issues an alarm, greatly reducing the labor intensity and personnel fatigue. The problem of the lag of manual monitoring is also solved, improving the management efficiency of the entire production line.

[0018] 3. Reduce maintenance costs and false alarms: Compared with sensors or traditional image technologies, the deep learning-based image recognition method is not affected by mechanical wear and environmental factors (such as dust, temperature, and light changes), has low maintenance costs, and the long-term stability and reliability of the system are significantly improved.

[0019] 4. Adapt to complex production environments: The YOLOv5 model has strong adaptability and generalization ability, and can handle complex and changeable production environments. Even in the case of light changes or perspective changes, the system can still operate stably; the high accuracy and low latency characteristics of the model ensure the stable and efficient operation of the system under various production conditions. At the same time, the YOLOv5 model architecture is flexible, easy to expand and maintain, and adapts to different production environments and requirements. Data augmentation and model tuning methods ensure the stable performance of the system in various complex scenarios.

[0020] 5. Achieve intelligent and automated production: The present invention realizes the intelligence and automation of material blockage detection and storage cabinet status monitoring by integrating multi-channel video stitching technology and deep learning models, improves the overall automation level of the production line, and reduces the frequency of manual intervention.

[0021] This intelligent solution not only improves production efficiency but also lays a foundation for the construction of future smart factories.

[0022] In summary, the present invention significantly improves the accuracy and efficiency of material blockage detection in the cigarette production and manufacturing process by combining advanced deep learning technology and multi-channel video monitoring, reduces labor costs and maintenance expenses, and provides reliable technical support for realizing efficient and intelligent production management. Brief Description of the Drawings

[0023] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0024] Figure 1 It is a schematic diagram of the application process of the present invention; Figure 2 It is a schematic diagram of the YOLOv5 model architecture of the present invention. Specific embodiments

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] Embodiment 1 This embodiment relates to a method for detecting and monitoring the state of blocked materials based on deep learning, including S1 Data collection and annotation Obtain blocked material videos captured by multiple cameras, and intercept the videos into static images at a rate of one second per frame; Manually annotate the intercepted images through the LabelImg tool, mark the blocked material scenes and normal material scenes therein, and form a preliminary data set; S2 Data augmentation and preprocessing Perform data augmentation processing on the annotated image data set; Each image is augmented by 5 times through rotation, flipping, blurring, adding noise, changing brightness, etc. to increase the diversity and robustness of the data set; Divide the augmented data set into a training set and a validation set according to a certain proportion; S3 Model optimization Referring to Figure 2 , based on the YOLOv5 model, perform image detection on the blocked situation of tobacco leaves or cut tobacco. To optimize the detection ability of small targets, the following optimizations and improvements are made to the YOLOv5 model; S3.1 Configure the Anchor module First, analyze the data set and count the size range of small targets in the samples of the data set; Then, in the corresponding configuration file of the YOLOv5 model, configure the Anchor module based on the size range of the small targets. At the same time, considering that both the conveyor belt and the storage cabinet in the detection scene are long-strip targets and usually have a large aspect ratio (for example, the height is much greater than the width, or the width is much greater than the height), in order to optimize the detection effect of such targets, the settings of the Anchor module are adjusted according to the aspect ratio of the targets while adjusting the size range; Example: Configuration parameters of the Anchors module: - [5, 6, 8, 14, 15, 11] # Detection of small target sizes; - [16, 8, 64, 16, 128, 32] # Anchor settings for horizontally long strip targets; - [8, 16, 16, 64, 32, 128] # Anchor settings for vertically long strip targets; S3.2 Introduce the CBAM attention module Since the detection screen is composed of multiple video streams stitched together, and more involves small target detection, the CBAM attention module is set at the early stage of the convolutional layer (especially the first layer or the first few layers) and after the upsampling layer, so as to help the model focus on the key areas of small targets at this stage, enhance the model's attention to important areas. Especially when there are small targets or local information is missing, the CBAM attention module helps the model focus on these areas through channel and spatial attention; For the detection of small targets (such as blocked materials), the YOLOv5 model needs to fuse feature maps of different scales to capture the targets. In the upsampled feature maps, different scale feature information has been fused. The CBAM attention module can further optimize these feature maps and highlight key details. For example, blocked materials usually show local anomalies. The spatial attention mechanism of CBAM can help the model accurately locate these small and possibly partially occluded targets in high-resolution images without being interfered by background noise.

[0027] S3.3 Add the Dice Loss function The loss function of the YOLOv5 model includes classification loss, regression loss, and Dice Loss; Dice Loss calculates the overlap degree between the predicted region and the real region, rather than relying solely on the intersection over union of the target boxes. This is very effective for long strip targets with fuzzy edges; for targets with irregular shapes and curved boundaries, Dice Loss can effectively optimize the model and improve the model's perception ability of edge details, avoiding errors caused by irregular target shapes.

[0028] When calculating the loss, Dice Loss is weighted and combined with the original classification loss and regression loss. For each predicted box, the overlapping region between the predicted box and the real box is calculated by Dice Loss, and the weight is adjusted according to the needs of model training to avoid affecting the training process of other loss terms; S4 Model training The optimized YOLOv5 model is used to train the model on the dataset, and the model is tuned by adjusting hyperparameters and optimization algorithms until the detection accuracy of the model on the validation set reaches the expected standard; After training is completed, the final weight file is generated; The training parameters of the YOLOv5 model are: batch size = 16, epochs = 100, learning rate = 0.001; After training is completed, the final weight file is generated, and the accuracy of the validation set reaches more than 95%.

[0029] S5 Real-time Detection and Alarm The real-time video stream of the virtual camera is input into the YOLOv5 model to detect the material blockage situation and storage cabinet status in the cigarette production line and output the results; When material blockage is detected, based on the set confidence threshold and alarm trigger mechanism, check whether its confidence is higher than 50%. If it is higher than 50%, an alarm will be triggered through a pop-up window on the GUI interface, and the alarm information will be recorded for the operator to handle in a timely manner.

[0030] As an improvement of the above embodiment, S1 data collection and annotation includes S11 Acquisition and Processing of Multi-channel Video Streams Multiple cameras installed on the top of the production site are used to collect video streams through the ONVIF protocol, and multiple video streams are combined into one video stream through a video decoder; S12 Virtual Camera Simulation The HDMI signal output by the video decoder is input into the PC through an HDMI video capture card and simulated as the output of a virtual camera for detection by the YOLOv5 model.

[0031] Specifically, the cameras are multiple rotating surveillance cameras installed in the underground leaf storage room of the cigarette making process to cover the storage cabinet, conveyor belt, and discharge port areas, and the cameras are connected to the video decoder through the ONVIF protocol.

[0032] Embodiment 2 Refer to Figure 1 , the application of the material blockage detection and status monitoring method based on deep learning described in Embodiment 1 includes Collect the material blockage video stream through multiple cameras, combine multiple video streams into one video stream through a video decoder, and then transmit it to the LED controller through the HDMI channel for display on the output Led screen; The HDMI signal output by the video decoder is input into the PC through an HDMI video capture card and simulated as the output of a virtual camera; Input the real-time video stream of the virtual camera into the improved YOLOv5 model to detect the material blockage situation and the storage cabinet status in the cigarette production line; Output the detection results in real time and display them.

[0033] Embodiment 3 The material blockage detection and storage cabinet status monitoring system based on the YOLOv5 model includes I. Hardware devices: 9 rotation monitoring cameras installed on the top; 1 video decoder; 1 HDMI video capture card; 1 PC, configured with an Intel Core i7 processor, 16GB RAM, and an AMD graphics card; 1 LED controller, model number NovaStar VX4S; 1 output screen with a resolution of 1920x1080; II. Software tools YOLOv5 model, PyTorch framework; LabelImg annotation tool.

[0034] Data augmentation program, including image rotation, flipping, blurring, adding noise, changing brightness, etc.

[0035] III. Specific layout and application steps of this system: 1. Video acquisition: Install 9 rotation monitoring cameras in the underground leaf storage area of the cigarette making process, covering the storage cabinet, conveyor belt, and discharge port areas. The cameras are connected to the video decoder through the ONVIF protocol.

[0036] The video decoder combines the video streams of multiple cameras into one video stream, transmits it to the LED controller through the HDMI channel, and displays it on the output screen.

[0037] 2. Data collection and annotation: Use the camera to collect the real-time video of the cigarette production line, and intercept the video into static images every second.

[0038] Use the LabelImg tool to manually annotate the intercepted images, mark the material blockage scenarios and normal material scenarios, and form a preliminary data set.

[0039] 3. Data augmentation and preprocessing: Use the data augmentation program to augment the annotated image data set, and augment the data set 5 times by rotating, flipping, blurring, adding noise, changing brightness, etc.

[0040] 4. Model Training and Tuning: Train using the improved YOLOv5 model on the augmented dataset, and set the training parameters as follows: batchsize = 16, epochs = 100, learning rate = 0.001.

[0041] After training is completed, generate the final weight file, and the accuracy of the validation set reaches over 95%.

[0042] 5. Model Deployment and Detection: Package the trained YOLOv5 model, the detection program, and its dependent running components into an executable file.

[0043] Input the video signal collected by the HDMI video capture card into the PC and simulate it as the output of a virtual camera for real-time detection by the YOLOv5 model.

[0044] Write a program to detect the output results of the model. When a blockage type is detected and the confidence level is higher than 50%, an alarm will be issued through a pop-up window on the GUI interface.

[0045] 6. Effect Data Detection Accuracy: 95% False Alarm Rate: 2% Response Time: <200ms It should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A blockage detection and status monitoring method based on deep learning, characterized by: include S1 Data collection and annotation Obtain the blocking videos shot by multiple cameras, and capture the videos into static images every second; Manually annotate the captured images, mark the scenes with blocked materials and scenes with normal materials, and form a preliminary data set; S2 data augmentation and preprocessing Perform data augmentation on the labeled image dataset; Divide the expanded data set into training set and validation set in proportion; S3 Model Optimization Based on the YOLOv5 model, image detection of tobacco leaf or tobacco plugging is performed. In order to optimize the small target detection capability, the YOLOv5 model is optimized and improved as follows; S3.1 Configure Anchor module First, the data set is analyzed to count the size range of small objects in the samples of the data set; Then, in the corresponding configuration file of the YOLOv5 model, configure the Anchor module based on the size range of the small target, and adjust the size range according to the aspect ratio of the target; S3.2 Introducing the CBAM attention module The CBAM attention module is set in the early stage of the convolution layer and after the upsampling layer to enhance the model's attention to important areas at this stage, especially for small targets or when local information is missing. The CBAM attention module helps the model focus on these areas through channel and spatial attention; S3.3 Add Dice Loss function The loss function of the YOLOv5 model includes classification loss, regression loss and Dice Loss; When calculating the loss, Dice Loss is weighted and combined with the original classification loss and regression loss. For each predicted box, the overlapping area between the predicted box and the true box is calculated by Dice Loss. The weight is adjusted according to the needs of model training to avoid affecting the training process of other loss items. S4 model training Use the optimized YOLOv5 model to train on the dataset, and fine-tune the model by adjusting the hyperparameters, using the transfer learning method until the detection accuracy of the model on the validation set reaches the expected standard; After training is completed, the final weight file is generated; S5 real-time detection and alarm Input the real-time video stream of the virtual camera into the YOLOv5 model to detect the blockage and storage cabinet status in the cigarette production line and output the results; When blockage is detected, based on the set confidence threshold and alarm trigger mechanism, check whether the confidence is higher than 50%. If it is higher than 50%, an alarm pops up and records the alarm information so that the operator can handle it in time.

2. The method for material blockage detection and state monitoring based on deep learning according to claim 1, characterized in that: The step S1 of data collection and annotation includes S11 Multi-channel video stream acquisition and processing Use multiple cameras installed on the top of the production site to collect the blocking video, and combine multiple video streams into one video stream through a video decoder; S12 virtual camera simulation The HDMI signal output by the video decoder is input to the PC through the HDMI video capture card and simulated as the output of a virtual camera for detection by the YOLOv5 model.

3. The method for material blockage detection and state monitoring based on deep learning according to claim 2, characterized in that: The cameras are multiple rotating monitoring cameras installed in the underground silk-making leaf storage room to cover the storage cabinets, conveyor belts and discharge port areas. The cameras are connected to the video decoder via the ONVIF protocol.

4. The method for material blockage detection and state monitoring based on deep learning according to claim 1, characterized in that: The training parameters of the YOLOv5 model are: batch size=16, epochs=100, learning rate=0.001; After the training is completed, the final weight file is generated, and the accuracy of the validation set reaches more than 95%.

5. The method for material blockage detection and state monitoring based on deep learning according to claim 1, characterized in that: The data augmentation method comprises Each image is enlarged 5 times by rotating, flipping, blurring, adding noise, changing brightness, etc. to increase the diversity and robustness of the dataset.

Images