Conveyor belt baking product system based on deep learning and counting method thereof

Abstract

The invention relates to the technical field of baking, in particular to a conveyor belt baking product system based on deep learning and a counting method thereof.The conveyor belt baking product system comprises data input, the output end of the data input is electrically connected with data acquisition, the output end of the data acquisition is electrically connected with target detection, and the output end of the target detection is electrically connected with target tracking; the output end of the target tracking is electrically connected with counting and output; by integrating four core modules of data acquisition, target detection, target tracking and counting and output, comprehensive, efficient and automatic detection and counting of baked products on a conveyor belt are realized, the application of the method not only greatly improves the counting efficiency of the baked products, but also remarkably enhances the intelligent management level of a production line, and the production efficiency is improved. And a solid foundation is laid for transformation, upgrading and intelligent development of the baking industry.

Description

Technical Field

[0001] The present invention relates to the field of baking technology, and in particular to a conveyor belt baking product system and a counting method thereof based on deep learning. Background Art

[0002] With the rapid development of the baking industry, automation and intelligent production have become the key to improving production efficiency, reducing costs and ensuring product quality. In the production process of baked products, conveyor belts are widely used in various production links to achieve continuous and efficient transmission of products. However, accurate counting of baked products on conveyor belts has always been a technical problem.

[0003] Traditional methods for counting baked products mainly rely on manual operation or simple mechanical devices. These methods have many shortcomings: manual counting is inefficient and easily affected by human factors such as fatigue and misjudgment, resulting in inaccurate counting results; mechanical devices often have difficulty adapting to the diversity of product shapes, sizes, and colors, and have poor adaptability to complex environments; In recent years, the rapid development of deep learning technology has provided new solutions for conveyor belt baked product counting. However, applying deep learning technology to conveyor belt baked product counting also faces some challenges. First, the shape, size, color and other characteristics of baked products vary greatly, which brings difficulties to the training and recognition of deep learning models. Secondly, deep learning usually adopts the method of frame extraction detection when detecting targets, which inevitably brings a potential problem-duplicate counting. Since baked products move continuously on the conveyor belt, frame extraction detection may cause the same product to be counted multiple times, thus affecting the accuracy of the counting. In addition, due to the high-speed operation of the conveyor belt, the real-time requirements of the counting algorithm are extremely high, and it is necessary to achieve fast response while ensuring accuracy; Therefore, we propose a conveyor belt baked product system and its counting method based on deep learning. Summary of the invention

[0004] The object of the present invention is to provide a conveyor belt baked product system and a counting method thereof based on deep learning to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A conveyor belt baking product system based on deep learning includes data input, the output end of the data input is electrically connected to data acquisition, the output end of the data acquisition is electrically connected to target detection, the output end of the target detection is electrically connected to target tracking, the output end of the target tracking is electrically connected to counting and output, and the data acquisition is electrically connected to a high-definition camera, intelligent parameter adjustment and high-speed continuous shooting / single capture.

[0006] A method for counting baked products on a conveyor belt based on deep learning, comprising: target detection, target tracking, counting and output, wherein the steps of target detection are: S1. Image Preprocessing In the image preprocessing stage, the raw images passed by the data acquisition module will first go through a series of carefully designed processing steps to make them meet the input requirements of the YOLOv7 algorithm. Specifically, the size of the image will be adjusted to the fixed size used in model training to ensure that the network can correctly interpret it. At the same time, the image will also be normalized, that is, the pixel values ​​will be scaled to a specific range; S2. Feature extraction YOLOv7 uses its carefully designed deep neural network structure, such as the CSPDarknet backbone network, to perform deep convolution operations on the preprocessed images and gradually extract features from low-level to high-level; S3. Object Detection In the target detection stage, YOLOv7 uses the multi-level feature map obtained in the feature extraction stage and the anchor box mechanism to predict the possible baked product targets in the image. It removes redundant bounding boxes with high overlap through the non-maximum suppression algorithm and retains the best detection results. S4. Classification and confidence assessment After detecting the target, YOLOv7 will also classify each target and evaluate its confidence. The algorithm will also give a confidence score that combines the confidence of the target's existence and the confidence of the category it belongs to; S5. Result output The object detection module will output the processed results in the form of a list, each list item contains the category information, location information and confidence score of the baked product.

[0007] As a preferred solution of the present invention, the target tracking comprises the following steps: S6. Initialize the target When the system is first started or a new batch of baked products is processed, the target tracking module receives the first frame image and its detection results from the target detection module. Then, for each detected baked product, the module automatically assigns a unique tracking identifier and records the initial position and appearance feature vector of each product. Then, the module initializes the Kalman filter for each tracked target and sets its initial state to prepare for subsequent prediction and state update. S7. DeepSORT tracking In the process of processing continuous frames, the target tracking module uses the DeepSORT algorithm to perform efficient target tracking. For each new frame of image, the module first extracts the appearance feature vectors of all baked products in the current frame through the deep learning model, and then uses the Kalman filter to predict the possible position of each tracked target in the current frame. Subsequently, the module calculates the similarity between the appearance features of the current frame detection result and the target in the tracking list, and combines the prediction results of the Kalman filter, and uses the Hungarian algorithm for efficient data association, thereby achieving accurate matching of the target; S8. Hypothesis Testing and Status Update After the target matching is completed, the module will perform hypothesis testing on the matching results to evaluate the reliability of the matching results. For the tracking target that has not successfully matched any detection results for multiple consecutive frames, the module will determine whether the target has been lost according to the tracking confidence threshold set by DeepSORT, and take corresponding processing measures accordingly, such as maintaining the tracking status and continuing to predict or marking it as lost and removing it from the tracking list; S9. Forecasting and Adjustment The module uses the Kalman filter to predict the position of each tracked target before processing each frame of image. When a new detection result arrives, the module compares the predicted position with the actual detection result and dynamically adjusts the parameters of the Kalman filter according to the error size to optimize its prediction performance. S10. Target status management and avoiding duplicate counting During the entire tracking process, the module will continuously update the status information of each baked product, including key parameters such as position and motion characteristics. When a target is detected to have left the field of view or has been blocked for a long time, the module will determine whether the target needs to be removed from the tracking list based on the DeepSORT tracking strategy. At the same time, when a target that re-enters the field of view is detected, the module will confirm whether it is a tracked target by comparing the tracking identifier, thereby ensuring that each baked product is counted only once during the entire processing process.

[0008] As a preferred solution of the present invention, the counting and outputting comprises the following steps: S11. An initialization operation is required to set up a separate counter for each category of baked products, and then continuously receive data from the target detection and target tracking modules. These data include the unique identifier of each baked product, its category, and current location information. At the same time, the module also uses the position change in the tracking information to confirm whether the product has completely passed the counting area, so as to accurately update the value of the counter; S12. After the counting is completed, in order to meet the needs of real-time monitoring of the production line, the module provides a real-time display function on the user interface. Users can intuitively see the current total count and the specific count of each category of baked products through the interface. In addition to the real-time display function, the module also supports the generation of detailed counting reports.

[0009] Compared with the prior art, the present invention has the following beneficial effects: 1. In the present invention, by integrating four core modules of data acquisition, target detection, target tracking, counting and output, comprehensive, efficient and automatic detection and counting of baked products on the conveyor belt are realized. The application of this method not only greatly improves the counting efficiency of baked products, but also significantly enhances the intelligent management level of the production line, laying a solid foundation for the transformation, upgrading and intelligent development of the baking industry. The data acquisition module adopts a high-quality camera to ensure the clarity and integrity of the image data, laying a solid foundation for subsequent target detection and target tracking. The module also has the ability to adjust intelligent parameters, can capture clear images under different lighting conditions, and improves the adaptability and stability of data acquisition.

[0010] 2. In the present invention, the target detection module is based on the advanced YOLOv7 algorithm, which realizes high-precision real-time detection of baked products. YOLOv7 can not only maintain high detection accuracy, but also maintain a fast processing speed, effectively coping with the challenges of high-speed conveyor belts and large amounts of image data. This technological breakthrough has greatly improved the automation level and efficiency of the production line.

[0011] 3. In the present invention, the target tracking module uses the DeepSORT algorithm to achieve accurate tracking and identification of baked products in consecutive frames. By assigning a unique identifier to each product and keeping track of it in consecutive frames, the module effectively avoids the problem of duplicate counting and improves the accuracy and reliability of counting.

[0012] 4. In the present invention, the counting and output module integrates the results of target detection and target tracking, and outputs the final counting results in an intuitive and detailed manner. The module not only supports real-time display function, which is convenient for users to monitor production conditions in real time, but also can generate detailed counting reports, which provides strong support for the optimization of enterprise production processes and sales management, and has achieved remarkable technical effects in improving the automation level of production lines, improving detection accuracy and efficiency, and avoiding repeated counting, providing an efficient and reliable solution for the production and management of baked products. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is a system framework diagram of the present invention;

[0014] Figure 2 This is a working diagram of the target detection module of the present invention; Figure 3 This is a working diagram of the target tracking module of the present invention; Figure 4 This is the first counting effect diagram of the present invention; Figure 5 This is the second counting effect diagram of the present invention. In the figure: 1. Data input; 2. Data acquisition; 3. Target detection; 4. Target tracking; 5. Counting and output. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0016] For examples, see Figure 1-Figure 4 , the present invention provides a technical solution: A conveyor belt baking product system based on deep learning includes a data input 1, an output end of the data input 1 is electrically connected to a data acquisition 2, an output end of the data acquisition 2 is electrically connected to a target detection 3, an output end of the target detection 3 is electrically connected to a target tracking 4, an output end of the target tracking 4 is electrically connected to a counting and output 5, and a high-definition camera, intelligent parameter adjustment, and high-speed continuous shooting / single capture are electrically connected in the data acquisition 2.

[0017] A method for counting baked products on a conveyor belt based on deep learning includes: target detection 3, target tracking 4, and counting and output 5. The steps of target detection 3 are: S1. Image Preprocessing In the image preprocessing stage, the original image passed by the data acquisition module will first go through a series of carefully designed processing steps to make it meet the input requirements of the YOLOv7 algorithm. Specifically, the size of the image will be adjusted to the fixed size used in model training to ensure that the network can parse it correctly. At the same time, the image will also be normalized, that is, the pixel value is scaled to a specific range such as 0 to 1 or -1 to 1, which helps to speed up the model convergence and improve detection accuracy; S2. Feature extraction YOLOv7 uses its carefully designed deep neural network structure, such as the CSPDarknet backbone network, to perform deep convolution operations on the preprocessed images and gradually extract features from low-level to high-level. Low-level features mainly focus on local details of the image, such as the edges and textures of baked products, which are crucial for accurately locating the bounding box. High-level features reflect more of the overall structure and category information of the product, such as shape, size, color distribution, etc., which are crucial for accurate classification. The YOLOv7 algorithm has shown extraordinary capabilities in the field of target detection. It can not only maintain high accuracy (the best accuracy can reach 56.8% AP) but also maintain a real-time detection speed of more than 30FPS. This high efficiency ensures that even in the face of high-speed conveyor belts and large amounts of image data, the system can quickly and accurately complete the detection task, greatly improving the efficiency and automation level of the production line; S3. Object Detection In the target detection stage, YOLOv7 uses the multi-level feature map obtained in the feature extraction stage and the anchor box mechanism to predict the possible baked product targets in the image. The non-maximum suppression (NMS) algorithm is used to remove redundant bounding boxes with high overlap and retain the best detection results. S4. Classification and confidence assessment After detecting the target, YOLOv7 will also classify each target and evaluate its confidence. The algorithm will also give a confidence score that combines the confidence of the target's existence and the confidence of the category it belongs to; S5. Result output The target detection module will output the processed results in the form of a list. Each list item contains the category information cls of the baked product, the coordinates pos of the positioning information bounding box, and the confidence score score. The category information (cls) indicates the type of baked product detected, such as bread, cake, biscuits, etc. The positioning information (pos) accurately points out the specific location of each baked product in the image. The form of pos is [x1, y1, x2, y2], where x1, y1, x2, and y2 represent the four coordinates of the positioning information respectively. The confidence score (score) is a quantitative expression of the algorithm's trust in the detection result. It comprehensively considers the probability of the target's existence and the accuracy of the category attribution, and provides an intuitive and quantitative evaluation standard for the credibility of the result. These list information provides key data support for subsequent target tracking and counting.

[0018] Target tracking 4 includes the following steps: S6. Initialize the target When the system is first started or a new batch of baked products begins to be processed, the target tracking module receives the first frame image and its detection results from the target detection module. Then, for each detected baked product, the module automatically assigns a unique tracking identifier ID. At the same time, it also records the initial position bounding box coordinates and appearance feature vector of each product extracted through the deep learning model. Then, the module initializes the Kalman filter for each tracked target and sets its initial state such as position, motion trajectory, etc., to fully prepare for subsequent prediction and state update. S7. DeepSORT tracking In the process of processing continuous frames, the target tracking module uses the DeepSORT algorithm to perform efficient target tracking. For each new frame of image, the module first extracts the appearance feature vectors of all baked products in the current frame through the deep learning model, and then uses the Kalman filter to predict the possible position of each tracked target in the current frame. Subsequently, the module calculates the similarity between the appearance features of the current frame detection result and the target in the tracking list, and combines the prediction results of the Kalman filter, and uses the Hungarian algorithm for efficient data association, thereby achieving accurate matching of the target; S8. Hypothesis Testing and Status Update After the target matching is completed, the module will perform hypothesis testing on the matching results to evaluate the reliability of the matching results. For the tracking target that has not successfully matched any detection results for multiple consecutive frames, the module will determine whether the target has been lost according to the tracking confidence threshold set by DeepSORT, and take corresponding processing measures accordingly, such as maintaining the tracking status and continuing to predict or marking it as lost and removing it from the tracking list; S9. Forecasting and Adjustment The module uses the Kalman filter to predict the position of each tracked target before processing each frame of image. When a new detection result arrives, the module compares the predicted position with the actual detection result and dynamically adjusts the parameters of the Kalman filter according to the error size to optimize its prediction performance. S10. Target status management and avoiding duplicate counting During the entire tracking process, the module will continuously update the status information of each baked product, including key parameters such as position and motion characteristics. When a target is detected to have left the field of view or has been blocked for a long time, the module will set the maximum lost frame number threshold according to the DeepSORT tracking strategy to determine whether the target needs to be removed from the tracking list. At the same time, when a target that re-enters the field of view is detected, the module will confirm whether it is a tracked target by comparing the tracking identifier, thereby ensuring that each baked product is counted only once during the entire processing process.

[0019] Counting and outputting 5 includes the following steps: S11. An initialization operation is required to set up an independent counter for each category of baked products, and then continuously receive data from the target detection and target tracking modules. These data include the unique identifier ID, category and current location information of each baked product. At the same time, the module also uses the position change in the tracking information to confirm whether the product has completely passed the counting area, so as to accurately update the value of the counter; S12. After completing the counting, in order to meet the needs of real-time monitoring of the production line, the module provides a real-time display function on the user interface. Users can intuitively see the current total count and the specific count of each category of baked products through the interface. In addition to the real-time display function, the module also supports the generation of a detailed counting report in txt format. The report contains the following information: Product ID: a unique identifier for each baked product, used to query product details in the database; Product category: classification information of baked products, which helps companies understand the sales of various products; Location information: records the specific location of the product on the production line or the time point when it passes through the counting area, providing a basis for optimizing the production process.

[0020] Workflow of the present invention: When the conveyor belt baked product system and counting method based on deep learning designed by the present invention are working, data acquisition 2 is responsible for real-time capture of images and video data of baked products on the conveyor belt, providing a basis for subsequent target detection 3 and target tracking 4. This module uses high-quality image acquisition equipment to ensure the clarity and integrity of the data. Target detection 3 is based on the deep learning algorithm YOLOv7 to process and analyze the collected image data. It can automatically learn and identify the characteristics of baked products, such as shape, size, color, etc., so as to accurately distinguish the category information, positioning information and confidence of each product. Target tracking 4 is another important component of the system. It uses an advanced inter-frame target tracking algorithm to continuously track and identify baked products in continuous frames. By assigning a unique identifier to each product and keeping track of them in continuous frames, this module ensures that each product is counted only once, avoiding the problem of repeated counting. Finally, counting and output 5 is responsible for integrating the output results of target detection 3 and target tracking 4, and generating the final counting result. This module uses an efficient computing framework and algorithm optimization technology to ensure that the counting results can be output quickly and accurately even when the conveyor belt runs at high speed.

[0021] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A conveyor belt baked product system based on deep learning, comprising a data input (1), characterized in that: The output end of the data input (1) is electrically connected to a data acquisition device (2), the output end of the data acquisition device (2) is electrically connected to a target detection device (3), the output end of the target detection device (3) is electrically connected to a target tracking device (4), the output end of the target tracking device (4) is electrically connected to a counting and output device (5), and the data acquisition device (2) is electrically connected to a high-definition camera, intelligent parameter adjustment, and high-speed continuous shooting / single-frame capture.

2. A conveyor belt baked product counting method based on deep learning, applicable to the conveyor belt baked product system based on deep learning as claimed in claim 1, characterized in that: include: Target detection (3), target tracking (4) and counting and output (5), the steps of target detection (3) are: S1. Image Preprocessing In the image preprocessing stage, the raw image passed by the data acquisition module will first go through a series of carefully designed processing steps to make it meet the input requirements of the YOLOv7 algorithm. Specifically, the image size will be adjusted to the fixed size used in model training to ensure that the network can correctly parse it. At the same time, the image will also be normalized, that is, the pixel value will be scaled to a specific range (such as 0 to 1 or -1 to 1); S2. Feature extraction YOLOv7 uses its carefully designed deep neural network structure, such as the CSPDarknet backbone network, to perform deep convolution operations on the preprocessed images and gradually extract features from low-level to high-level; S3. Object Detection In the target detection stage, YOLOv7 uses the multi-level feature map obtained in the feature extraction stage and the anchor box mechanism to predict the possible baked product targets in the image. The non-maximum suppression (NMS) algorithm is used to remove redundant bounding boxes with high overlap and retain the best detection results. S4. Classification and confidence assessment After detecting the target, YOLOv7 will also classify each target and evaluate its confidence. The algorithm will also give a confidence score that combines the confidence of the target's existence and the confidence of the category it belongs to; S5. Result output The object detection module outputs the processed results in the form of a list. Each list item contains the category information (cls), positioning information (coordinates of the bounding box pos) and confidence score (score) of the baked product.

3. The method for counting baked products on a conveyor belt based on deep learning according to claim 2, characterized in that: The target tracking (4) comprises the following steps: S6. Initialize the target When the system is first started or a new batch of baked products is processed, the target tracking module receives the first frame image and its detection results from the target detection module. Then, for each detected baked product, the module automatically assigns a unique tracking identifier (ID), and also records the initial position (bounding box coordinates) and appearance feature vector (extracted by the deep learning model) of each product. Then, the module initializes the Kalman filter for each tracked target and sets its initial state (such as position, motion trajectory, etc.) to prepare for subsequent predictions and state updates. S7. DeepSORT tracking In the process of processing continuous frames, the target tracking module uses the DeepSORT algorithm to perform efficient target tracking. For each new frame of image, the module first extracts the appearance feature vectors of all baked products in the current frame through the deep learning model, and then uses the Kalman filter to predict the possible position of each tracked target in the current frame. Subsequently, the module calculates the similarity between the appearance features of the current frame detection result and the target in the tracking list, and combines the prediction results of the Kalman filter, and uses the Hungarian algorithm for efficient data association, thereby achieving accurate matching of the target; S8. Hypothesis Testing and Status Update After the target matching is completed, the module will perform hypothesis testing on the matching results to evaluate the reliability of the matching results. For the tracking target that has not successfully matched any detection results for multiple consecutive frames, the module will determine whether the target has been lost according to the tracking confidence threshold set by DeepSORT, and take corresponding processing measures accordingly, such as maintaining the tracking status and continuing to predict or marking it as lost and removing it from the tracking list; S9. Forecasting and Adjustment The module uses the Kalman filter to predict the position of each tracked target before processing each frame of image. When a new detection result arrives, the module compares the predicted position with the actual detection result and dynamically adjusts the parameters of the Kalman filter according to the error size to optimize its prediction performance. S10. Target status management and avoiding duplicate counting During the entire tracking process, the module will continuously update the status information of each baked product, including key parameters such as position and motion characteristics. When a target is detected to have left the field of view or has been blocked for a long time, the module will determine whether the target needs to be removed from the tracking list based on DeepSORT's tracking strategy (setting the maximum lost frame number threshold). At the same time, when a target that re-enters the field of view is detected, the module will confirm whether it is a tracked target by comparing the tracking identifier, thereby ensuring that each baked product is counted only once during the entire processing process.

4. The method for counting baked products on a conveyor belt based on deep learning according to claim 2, characterized in that: The counting and outputting (5) comprises the following steps: S11. An initialization operation is required to set up a separate counter for each category of baked products, and then continuously receive data from the target detection and target tracking modules. These data include the unique identifier (ID) of each baked product, its category, and current location information. At the same time, the module also uses the location change in the tracking information to confirm whether the product has completely passed the counting area, so as to accurately update the value of the counter; S12. After completing the counting, in order to meet the needs of real-time monitoring of the production line, the module provides a real-time display function on the user interface. Users can intuitively see the current total count and the specific count of each category of baked products through the interface. In addition to the real-time display function, the module also supports the generation of detailed counting reports (txt format).

Images