Conveyor belt bottle cap counting system based on deep learning and multi-target tracking

Abstract

The invention relates to the technical field of industrial automation and machine vision, and discloses a conveyor belt bottle cap counting system based on deep learning and multi-target tracking. The BGR image frame output module is used for realizing equipment enumeration, packet length optimization, closing triggering, continuous stream taking and unified conversion of multiple pixel formats through a lightweight packaged Haikang GigE camera SDK, and outputting a stable BGR image frame; the detection and tracking module is used for loading a pre-trained YOLO model to carry out bottle cap target detection on the image frame, the bottle cap target is detected by using the YOLO model, a unique track id is allocated to each detection target through a multi-target tracking algorithm, and an overline counting strategy based on the history of a horizontal reference line and a target center point is introduced on this basis, so that the bottle cap target detection accuracy is improved. A count event is triggered only when the target center point crosses the virtual count line between adjacent image frames.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial automation and machine vision, and in particular to a conveyor belt cap counting system based on deep learning and multi-target tracking. BACKGROUND

[0002] In the industrial production line, there is a general need for online statistics and process monitoring of the number of bottle caps in the processes of filling, capping and packaging. As the counting object, bottle caps have the characteristics of small volume, various shapes and obvious material differences. They can be made of plastic or metal, and can be transparent or translucent. In the actual production process, bottle caps are usually transported at high speed on the conveyor belt, and there are situations such as inconsistent spacing, random posture, parallel or stacking passing, etc., accompanied by complex working conditions such as surface highlight reflection, environmental light fluctuation, vibration and ghosting, chain speed mutation, background texture and foreign object interference, etc. These factors together make it challenging to achieve reliable counting while ensuring stability, accuracy and low latency.

[0003] With the gradual popularization of deep learning technology in the field of industrial vision, the counting scheme based on target detection network (such as YOLO series) for frame-level identification of bottle caps and combined with multi-target tracking algorithm to maintain the consistency of target timing has gradually become the mainstream. This kind of scheme usually realizes cross-line unique counting based on detection and tracking by arranging a virtual counting line or counting area, and has stronger robustness than traditional methods in complex background conditions. However, in the actual engineering landing process, this kind of scheme still faces several common problems. First, in the image acquisition link, the GigEVision camera commonly used in industrial sites involves device enumeration, network packet length optimization, trigger mode configuration, continuous streaming and memory buffer management, and conversion of various pixel formats (such as Mono8, BGR8, Bayer format) in the SDK integration process. Improper configuration of any link may cause frame rate fluctuations, frame loss or decoding abnormalities, thereby affecting the stability of subsequent tracking and the reliability of counting. Secondly, in terms of real-time and interface interaction, the detection and tracking inference has high requirements for computing power and scheduling. If the inference calculation and human-machine interface update are in the same thread or the scheduling strategy is unreasonable, it is easy to cause interface lag or inference blockage, reducing the system usability and the operator's perception ability of the running state.

[0004] In addition, during actual installation and operation, problems such as lens shading, camera black border or excess field of view entering the picture often occur, and if there is a lack of software-level cropping and area management, a large amount of irrelevant background noise will be introduced, which not only increases the false detection probability, but also increases the reasoning calculation burden. At the same time, during multi-target tracking, tracking ID switching may occur when the target is blocked, the speed changes or short-time detection fails, and if there is a lack of deduplication and anti-shake strategy based on historical trajectory, repeated counting or rollback problems may occur during cross-line counting; for abnormally large size detection boxes, background false detections or abnormal frames, if effective filtering is not performed, the stability of the counting result will be further affected.

[0005] At the operation and maintenance level, an intuitive man-machine interface is usually required for parameter adjustment and state monitoring in an industrial field, and key parameters need to be persistently saved to maintain consistent operation behavior after shift change or system restart. At the same time, reasoning time visualization, minute-level statistics, counting clearing and other functions need to be provided to align the production line beat, locate problems and trace quality. However, some existing systems either ignore the interface and statistical capabilities in engineering implementation or lack configuration persistence and visual adjustment means, resulting in difficulties in field operation and system migration. SUMMARY

[0006] The purpose of the present application is to solve the problems existing in the prior art, and to provide a conveyor belt bottle cap counting system based on deep learning and multi-target tracking.

[0007] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: The conveyor belt bottle cap counting system based on deep learning and multi-target tracking comprises: An image acquisition module is used to realize device enumeration, packet length optimization, trigger off, continuous streaming and unified conversion of multiple pixel formats through a lightweight packaged Hikvision GigE camera SDK, and output stable BGR image frames; A detection and tracking module is used to load a pre-trained YOLO model to detect bottle cap targets in image frames, and assign a unique track id to each detection target by combining a multi-target tracking algorithm to maintain temporal consistency; A counting logic module is used to set a horizontal reference line as a virtual counting line for counting in the image frame, and maintain the y-coordinate history of the center point of each target with a track id, trigger a counting event when the center point of the target crosses the virtual counting line between adjacent image frames, and deduplicate through the set of counted ids to avoid repeated counting, and obtain the counting result; A man-machine interface module is used to display image frames and counting results during processing, and is used for periodic refreshing of the picture and triggering of detection and tracking processing to display reasoning time information and statistical information. The configuration management module is configured to persistently save the image cropping parameters and the positions of the virtual counting lines through a configuration file, and load the parameters during system startup or running to realize parameter recovery.

[0008] Preferably, the plurality of pixel formats include Mono8, BGR8, and BayerRG8.

[0009] Preferably, the human-computer interface module is configured to receive an adjustment operation of the image cropping parameters and the positions of the virtual counting lines input through the human-computer interface, and transmit the adjusted parameters to the configuration management module to save the parameters by the configuration management module.

[0010] Preferably, the human-computer interface module is built into a full-screen interface by using PyQt5.

[0011] Preferably, the system further comprises a candidate filtering module configured to filter the detection boxes in the bottle cap target detection result based on the area proportion of the current image frame after the bottle cap target detection is completed, and eliminate oversized detection boxes or background false detections exceeding a set threshold, so as to reduce the probability of large-area shadows and equipment components being mistaken for bottle caps.

[0012] Preferably, the human-computer interface module is configured to provide a picture adjustment mode to visually preview left and right cropping of the image frames in the processing process, and transmit the obtained image cropping parameters to the configuration management module.

[0013] Preferably, the human-computer interface module further provides a counting zero function configured to reset the counting result generated by the counting logic module and the counted id set used for deduplication when the zero operation is performed.

[0014] Preferably, the system has control and abnormal handling behaviors during running, which are configured to: explicitly release the camera resources and the timing trigger mechanism when the state is switched from starting, stopping, and exiting; perform protection processing on the processes related to image acquisition and processing when stream acquisition failure or empty frame occurs; provide prompt information through the human-computer interface when the cropping parameters result in invalid image frames for processing.

[0015] The method comprises the following steps: The device enumeration, packet length optimization, trigger off, continuous stream acquisition, and unified conversion of a plurality of pixel formats are realized by using a lightweight packaged Hikvision GigE camera SDK to output stable BGR image frames. Load the pre-trained YOLO model to detect the bottle cap target in the image frame, and combine the multi-target tracking algorithm to assign a unique track id to each detection target to maintain the time sequence consistency; Set a horizontal reference line as a virtual counting line for counting in the image frame, and maintain the y coordinate history of the center point of each target with track id, trigger a counting event when the target center point crosses the virtual counting line between adjacent image frames, and remove the duplicate counting through the set of counted ids to avoid repeated counting, and obtain the counting result; Display the image frame in the processing process and the counting result; The image cropping parameters and the position of the virtual counting line are persistently saved through a configuration file, and the parameters are loaded during system startup or running to realize parameter recovery.

[0016] The beneficial effects of the present application are: The present application combines the bottle cap target detection based on deep learning with the multi-target tracking technology, and constructs a unique counting mechanism based on "detection-tracking-cross-line de-duplication". The system uses the YOLO model to detect the bottle cap target, and assigns a unique track id to each detection target through the multi-target tracking algorithm, and introduces a cross-line counting strategy based on the horizontal reference line and the history of the target center point, only when the target center point crosses the virtual counting line between adjacent image frames, a counting event is triggered, and the de-duplication processing is combined with the set of counted ids, which logically avoids the same target from being repeatedly counted when it is shaking, backtracking or changing id in a short time, thereby ensuring the monotonicity and non-backtracking of the counting process.

[0017] The present application provides a software-based visual cropping and ROI management method, which visually previews the left and right cropping of the image frame in the processing process through the picture adjustment mode, and persistently saves the corresponding image cropping parameters to the configuration file, quickly constructs a stable processing area in a software manner without changing the camera hardware parameters, effectively reduces the influence of image edge black edges, distortion areas and background noise on detection and tracking processing, thereby improving the stability and accuracy of bottle cap target detection and counting.

[0018] The present application performs lightweight packaging on the Hikvision GigE camera SDK, realizes device enumeration, packet length optimization, trigger configuration and continuous stream taking, and uniformly converts image frames of multiple pixel formats, supports uniform conversion of Mono8, BGR8, BayerRG8 and other formats into BGR image frames, ensures that the image data can be directly used by the subsequent processing process, reduces the decoding failure, frame loss or stream jitter problem caused by inconsistent pixel formats, and improves the stability of system running from the source.

[0019] In terms of operation scheduling, the application adopts a scheduling mode combining timing triggering and asynchronous inference calling to achieve a balance between low delay and smoothness of interface display in continuous image stream processing in a single machine running environment. The inference time consumption is displayed in real time in the interface, and the counting results are displayed in combination with a time sliding window-based statistical mode and a statistical list aggregated by minutes, so that the system can not only provide accumulated counting results, but also reflect the distribution of counting changes over time, meet the production line beat observation demand, and facilitate alignment with external statistical systems or team reports in the time dimension, avoiding information gaps in quality traceability scenarios caused by only providing accumulated totals without time resolution.

[0020] Under complex working conditions, the application introduces a candidate filtering mechanism based on the area proportion of image frames to eliminate abnormal large size detection boxes or background false detections in the detection results, reducing the situation that large area shadows, equipment components and other non-bottle cap regions are mistakenly considered as targets. In complex scenes such as high-speed conveying, dense stacking, occlusion overlapping, light changes and background interference, the "detection-tracking-cross-line deduplication" counting mechanism can significantly reduce the false detection rate and the missed detection rate, ensuring the stability and reliability of the counting results, and the counting accuracy can reach more than 99%.

[0021] The application introduces perfect control and abnormal handling behavior during system operation. In the event of abnormal situations such as failure to take flow, empty frame, or improper setting of cropping parameters resulting in invalid image frames for processing, the application performs protection processing on the related processing flow and provides prompt information through the human-machine interface. When the system is in state switching such as starting, stopping or exiting, the camera resources and timing trigger related resources are explicitly released, thereby reducing the risk of freezing or abnormal restart caused by resource unreleased during long-term operation, and improving the overall stability and reliability of the system.

[0022] In addition, the application adopts a modular software architecture design to decouple camera acquisition, inference model and human-machine interface logic, facilitating subsequent model replacement, multi-camera expansion, multi-line counting expansion and function enhancement, improving the scalability and maintainability of the system. By replacing hardware adjustment with software cropping and supporting rapid recovery and shift operation through parameter persistence, the application reduces the cost of on-site deployment and maintenance and improves the adaptability of the system in different production line environments. The application does not require modification of the existing production line structure or addition of complex mechanical devices, and can achieve stable and reliable online counting through software upgrade, providing an economic and efficient data basis for production monitoring, yield analysis and traceability management. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 The running flowchart of the conveyor belt bottle cap counting system based on deep learning and multi-target tracking proposed by the embodiment of the application; Figure 2The flow chart of the detection and tracking module of the conveyor belt bottle cap counting system based on deep learning and multi-target tracking provided in the embodiment of the present application is shown in the figure; Figure 3 The system block diagram of the conveyor belt bottle cap counting system based on deep learning and multi-target tracking provided in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to have a clearer understanding of the technical features, objectives and beneficial effects of the present application, the technical solutions of the present application are described in detail as follows, but it should not be understood as a limitation on the implementable scope of the present application. Unless specifically stated, the methods used in the present application are conventional methods in the technical field. In the present application, the materials, reagents or instruments used are not specified by the manufacturer, and are conventional products that can be obtained by market purchase.

[0025] Specifically, in one preferred embodiment provided by the present application, the conveyor belt bottle cap counting system based on deep learning and multi-target tracking comprises: The image acquisition module 100 is used to realize device enumeration, packet length optimization, trigger-off, continuous streaming and unified conversion of multiple pixel formats through the lightweight packaging of the Hikvision GigE camera SDK, and output stable BGR image frames.

[0026] In the embodiment of the present application, device enumeration, packet length optimization, trigger-off, continuous streaming and unified conversion of multiple pixel formats are realized through the lightweight packaging of the Hikvision GigE camera SDK, and stable BGR image frames are output.

[0027] In the embodiment of the present application, the system performs lightweight packaging on the Hikvision GigE camera SDK through hikvision.py, completes camera device enumeration, optimal packet length configuration and trigger mode off in the initialization stage, and maintains continuous streaming state during running. At the same time, the image acquisition module 100 performs unified conversion on the collected image frames of multiple pixel formats, including Mono8, BGR8 and BayerRG8, to output BGR image frames uniformly. Through the above processing, the output BGR image frames can be directly used by OpenCV and subsequent inference engines, so as to maintain the consistency and stability of image output under different sensor types and link conditions, and avoid decoding abnormalities or unstable streaming caused by inconsistent image formats.

[0028] Specifically, in terms of camera parameter configuration, it is recommended to set the camera resolution to 1280x720 or higher, the image acquisition frame rate to about 30 frames per second, the exposure time to be automatically adjusted according to the on-site lighting conditions, and the data packet length of the camera to be set to about 8000 bytes to optimize the image data transmission efficiency under the GigE interface.

[0029] In terms of system performance, running tests were conducted on a standard industrial control computer platform equipped with an Intel i5 processor and 8GB of memory. In this hardware environment, the processing delay of a single frame of image of the system was less than 50 milliseconds, the average CPU occupancy was about 40%, the memory occupancy was stable within 2GB, and the system could continuously run for 24 hours without abnormal interruption.

[0030] In terms of system expansion capability, the system provides an interface for integration with a manufacturing execution system or a programmable logic controller, supports real-time uploading of counting data and related process control functions, so as to access the cap counting results into the existing production management system.

[0031] Further, the conveyor belt cap counting system further comprises: The detection and tracking module 200 is used to load a pre-trained YOLO model to perform cap target detection on image frames, and combine a multi-target tracking algorithm to assign a unique track id to each detection target to maintain temporal consistency.

[0032] In the embodiment of the present application, around the technical requirements of counting accuracy and stability, the detection and tracking module 200 loads the YOLO model file bestcap.pt trained based on the Ultralytics framework, performs cap target detection on the image frames output by the image acquisition module, and combines a multi-target tracking algorithm to assign a corresponding track id to each detected cap target, thereby maintaining the consistency of target identification between consecutive image frames.

[0033] Specifically, the YOLO model used in the present application is trained based on the Ultralytics framework, and the training data set covers multiple cap types, including plastic caps, metal caps and transparent caps, and includes different light conditions and occlusion scenes. In order to improve the generalization ability of the model, data enhancement processing is introduced during model training, and operations such as rotation, scaling and brightness adjustment are performed on the training samples. The input size of the model is set to 640x640, the training round is 300 epochs, and SGD is used as the optimizer during the training process.

[0034] Further, the conveyor belt cap counting system further comprises a candidate filtering module, which is used to filter the detection boxes in the cap target detection result based on the area ratio of the current image frame after the cap target detection is completed, and eliminate the super-large detection boxes or background false detections that exceed the set threshold, thereby reducing the probability of large-area shadows and equipment components being mistaken for caps.

[0035] In the embodiments of the present application, in order to inhibit the interference of the abnormal area on the counting process, the system introduces candidate filtering processing based on the area proportion of the image frame after completing the cap target detection and obtaining the corresponding detection frame. Specifically, the candidate filtering module screens the detection results according to the area proportion of the detection frame in the current image frame, and when the area proportion of the detection frame exceeds a preset threshold, the detection frame is determined as an abnormal detection result and is removed. Through the above processing, the false detection situation caused by the background factors such as large-area shadow, conveyor belt structure or equipment components can be effectively filtered, thereby reducing the possibility that the non-cap target is mistakenly considered as a cap and participates in counting, and making the subsequent multi-target tracking and counting logic only process the candidate targets meeting the size constraint.

[0036] Further, the conveyor belt cap counting system further comprises: The counting logic module 300 is used for setting a horizontal reference line as a virtual counting line for counting in the image frame, maintaining the y-coordinate history of the center point of each target with a track id, triggering a counting event when detecting that the center point of the target crosses the virtual counting line between adjacent image frames, and de-duplicating through the set of counted ids to avoid repeated counting, thereby obtaining a counting result.

[0037] In the embodiments of the present application, the counting logic module 300 adopts a cross-line unique counting strategy based on the horizontal reference line and the history of the center point of the target. In this strategy, the system sets a virtual counting line for counting in the image frame, and continuously maintains the y-coordinate history of the center point of each target assigned with a track id by the detection and tracking module. When detecting that the center point of a target crosses the virtual counting line between adjacent image frames, the counting logic module 300 triggers a counting event once, and records the corresponding track id into the set of counted ids. Subsequently, in the subsequent image frame processing, if the track id already existing in the set of counted ids occurs cross-line behavior again, a new counting event is not triggered, thereby avoiding the same target from being repeatedly counted in the case of jitter, temporary back-off or track id re-binding for a short time. The position of the virtual counting line can be visually presented through the human-computer interface module, so as to be set and adjusted according to the actual conveyor belt scene. Through the above counting logic processing, the system can stably output the corresponding counting result in the continuous running process.

[0038] Further, the conveyor belt cap counting system further comprises: The human-computer interface module 400 is used for displaying the image frame and the counting result in the processing process, and is used for periodically refreshing the screen and triggering the detection and tracking processing to display the inference time consumption information and the statistical information.

[0039] In the embodiment of the present application, in view of the application scene requirement of coexistence of real-time and resource constraints, the system adopts a scheduling mode combining timing trigger and asynchronous inference calling in the human-computer interface module 400 to balance the processing delay of continuous image stream and the fluency of interface display in a single machine running environment. Through the scheduling mode, the detection and tracking processing is executed asynchronously in the background, and the human-computer interface module 400 refreshes and displays the image frames in the processing process according to the set period, thereby avoiding the blocking of interface response caused by inference calculation. At the same time, the time consumed by the inference process is recorded in real time and presented in the interface, which facilitates the observation of the current running state. Further, the human-computer interface module 400 combines the statistical method based on the sliding time window and the record list summarized by minutes to statistically display the counting result, so that the counting information can not only reflect the cumulative result, but also reflect the distribution over time. Through the above interface display mode, the system can meet the running monitoring requirement of production line beat observation, and at the same time, it is convenient to align the time dimension with the statistical report generated by the external system, thereby avoiding the problem of insufficient information in the quality traceability scene caused by only providing cumulative counting without time resolution.

[0040] The human-computer interface module is used for receiving the image cropping parameter and the adjustment operation of the virtual counting line position input through the human-computer interface, and transmitting the adjusted parameter to the configuration management module for saving the parameter by the configuration management module; the human-computer interface module adopts PyQt5 to build a full-screen interface.

[0041] The human-computer interface module is used for providing a picture adjustment mode to visually preview the left and right cropping of the image frame in the processing process, and transmitting the obtained image cropping parameter to the configuration management module. In the embodiment of the present application, considering that black edges, covered edges and field redundancy often occur in the image acquisition process, the human-computer interface module provides a picture adjustment mode for left and right cropping to visually preview the cropping range of the image frame in the processing process. Through the picture adjustment mode, the operator can perform software cropping on the effective area of the image frame without changing the camera parameter setting, and can observe the cropping effect in real time, so as to quickly determine the area range suitable for subsequent detection and tracking processing. After receiving the cropping operation, the human-computer interface module transmits the corresponding image cropping parameter to the configuration management module, and the configuration management module persistently saves the image cropping parameter and loads it in the subsequent running process. In this way, a stable processing area can be built at the software level, the interference of the image boundary area on the subsequent bottle cap target detection and multi-target tracking processing is reduced, and at the same time, the system parameter adjustment process is completed in a visual way, which is convenient for on-site deployment and daily maintenance.

[0042] The human-computer interface module also provides a count clear function for resetting the count result generated by the count logic module and the counted id set for deduplication when a clear operation is performed.

[0043] Further, the conveyor belt bottle cap counting system also comprises: The configuration management module 500 is configured to persistently save the image cropping parameters and the position of the virtual count line through a configuration file, and load the parameters during system startup or running to realize parameter recovery.

[0044] In the embodiment of the present application, the configuration management module 500 is configured to uniformly manage the image cropping parameters and the position of the virtual count line involved in the system running. After the human-computer interface module receives an adjustment operation on the image cropping parameters or the position of the virtual count line, the configuration management module 500 writes the corresponding parameters into the configuration file for persistent saving. When the system needs to reload the parameters during startup or running, the configuration management module 500 reads the saved image cropping parameters and the position of the virtual count line from the configuration file and applies them to the current running state, so that the system can be restored to the parameter settings consistent with the last running after restarting or state switching. Through the above configuration management mode, repeated configuration operations can be avoided, and the consistency of the image cropping area and the count reference line settings in the continuous running process can be ensured.

[0045] Further, the conveyor belt bottle cap counting system has control and abnormal processing behaviors during running, configured to: Explicitly release camera resources and timing trigger mechanism during state switching of starting, stopping and exiting; When stream acquisition failure or empty frame occurs, protection processing is performed on the image acquisition and processing related processes; When the cropping parameters cause the processing image frame to be invalid, a prompt message is provided through the human-computer interface.

[0046] In the embodiment of the present application, on the abnormal path of system operation, the software sets a protection processing mechanism for the case of flow taking failure and empty frame, and when the above abnormal state is detected, the related processing flow is returned in advance to avoid invalid images continuing to participate in subsequent detection and counting processing. At the same time, when the image frame used for processing is invalid due to improper setting of the image cropping parameter, the human-computer interface module provides prompt information to the operator in a timely manner so as to adjust the related parameter in time. When the system running state is switched, such as starting, stopping or exiting, the system explicitly releases the camera resources and the timing trigger related resources, thereby reducing the risk of running abnormity caused by the fact that the driving layer resources are not released in time. Further, in the scene of point inspection, shift change or re-starting counting, the system provides a counting zero function, synchronously resets the counting result generated by the counting logic module and the data state related to the counting process, and cooperates with the statistical information display update on the interface side, so as to ensure the continuity and integrity of the counting process between different running stages.

[0047] Specifically, in terms of fault processing, when the system detects that the empty frame or the flow taking failure occurs continuously, the system automatically reinitializes the camera and records the abnormal situation in the log; when the inference process is detected to be abnormal, the system performs rollback processing, switches to the backup model or prompts the operator to check the model file, so as to ensure that the system still has basic running ability under abnormal conditions.

[0048] Specifically, in one preferred embodiment provided by the present application, a conveyor belt bottle cap counting method based on deep learning and multi-target tracking, the method comprises: A Hikvision GigE camera SDK with light encapsulation is used to realize device enumeration, packet length optimization, trigger off, continuous flow taking and unified conversion of multiple pixel formats, and output stable BGR image frames; A pre-trained YOLO model is loaded to detect bottle cap targets in the image frames, and a multi-target tracking algorithm is combined to assign a unique track id to each detection target, so as to maintain the time sequence consistency; A horizontal reference line is set as a virtual counting line for counting in the image frame, and the y coordinate history of the center point of each target with a track id is maintained, a counting event is triggered when the center point of the target crosses the virtual counting line between adjacent image frames, and the set of counted ids is used for deduplication to avoid repeated counting, so as to obtain the counting result; The image frames in the processing process and the counting result are displayed; The image cropping parameter and the position of the virtual counting line are persistently saved through a configuration file, and the parameters are loaded during system startup or operation to realize parameter recovery.

[0049] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0050] Example 1: Application in a bottle cap counting scenario on a beverage manufacturing company's bottling production line. The system is deployed on the conveyor belt section at the rear of the bottling machine to perform online counting of the number of bottle caps passing through the conveyor belt.

[0051] The hardware configuration in this embodiment includes a Hikvision GigE camera with a resolution of 2 megapixels, supplemented by a ring LED light source. An industrial control computer serves as the operating platform, employing an Intel i5 processor and equipped with 8GB of memory. On the software side, the conveyor belt bottle cap counting system described in this invention is installed on the industrial control computer, and the pre-trained YOLO v5 model file bestcap.pt is loaded.

[0052] During system operation, the camera acquires image data at a frame rate of 30 frames per second, encapsulates and outputs BGR image frames through the camera SDK, and inputs these image frames to the detection and tracking module for bottle cap target detection and multi-target tracking. The counting logic module sets a virtual counting line at the center of the interface for counting. The system tracks the bottle cap target in consecutive image frames and triggers a counting event based on the relative position of the target's center point and the virtual counting line. Operators can adjust the cropping area through the human-machine interface module to eliminate the influence of the conveyor belt edge area on the detection and counting process.

[0053] The results show that under high-speed operation, with bottle caps passing through at a speed of approximately 5 to 10 per second and with a certain degree of stacking, the system maintains stable counting performance, with a counting accuracy consistently above 99.5%. The system interface displays the real-time inference time for each frame, with an average inference time of approximately 15 milliseconds. It also generates a count list every minute, and the statistical results are consistent with the production line PLC reports.

[0054] By applying the conveyor belt bottle cap counting system described in this invention, online counting is achieved without machine downtime or direct contact with the bottle caps, replacing the original counting scheme based on photoelectric sensors and reducing missed and duplicate counts. In this embodiment, the system's maintenance costs are significantly reduced during long-term operation, decreasing by 60%.

[0055] Example 2: Bottle cap counting scenario applied in a pharmaceutical packaging production line. Compared to beverage production lines, bottle caps in this type of production line are mainly made of transparent materials and come in a variety of sizes. Traditional photoelectric sensors are prone to high false detection rates and insufficient stability in this application scenario.

[0056] In this embodiment, the system employs a multi-camera synchronous acquisition method for image acquisition, using two Hikvision GigE cameras to cover different areas on the conveyor belt. The software system supports parallel processing of multi-channel images. To address the imaging characteristics of transparent bottle caps, the system retrains the bottle cap target detection model and adjusts the detection box area ratio threshold in the candidate filtering module during deployment. This threshold is set to approximately 0.1% to adapt to the size characteristics of the transparent bottle cap in the image.

[0057] After deployment and operation, the system maintained stable detection and counting performance despite changes in lighting conditions and the reflectivity of transparent bottle caps. Results showed that the counting accuracy improved from approximately 90% with traditional counting methods to approximately 98.5%. The system displays the counting results through minute-by-minute statistical information, facilitating rapid identification of production line fluctuations.

[0058] By applying the conveyor belt bottle cap counting system described in this invention, the reliability of bottle cap counting results in pharmaceutical packaging scenarios is improved, providing more detailed data support for quality traceability. Simultaneously, the human-machine interface module's visual display and parameter adjustment functions facilitate quick adjustments by operators based on actual operating conditions, thereby reducing production line downtime caused by counting anomalies.

[0059] This invention addresses the instability and proneness to miscounting or undercounting in existing online bottle cap counting processes under complex conditions. It provides a software-centric online bottle cap counting system that achieves stable, accurate, and low-latency counting of bottle caps under complex background conditions without altering the existing production line structure or camera hardware configuration. The system lightweightly encapsulates the Hikvision GigE camera SDK, sequentially performing device enumeration, optimal packet length setting, trigger shutdown, continuous streaming, and unified image pixel format conversion. This ensures the stability of the image acquisition process from the source, allowing the output image frames to be directly used in subsequent processing.

[0060] During the image processing stage, the system loads a YOLO model trained based on the Ultralytics framework to detect bottle cap targets in image frames. A multi-target tracking algorithm is then used to assign a unique trackid to each detected bottle cap target to maintain temporal consistency across consecutive image frames. Furthermore, the system arranges virtual counting lines in the image frames and maintains the y-coordinate history of the center point for each target with a trackid. When a target's center point is detected crossing the virtual counting line between adjacent image frames, this serves as the unique counting trigger condition. Simultaneously, the counting process is deduplicated by maintaining a set of already counted IDs, logically suppressing duplicate counting caused by target jitter, brief backtracking, or temporary changes in trackid.

[0061] To further improve the stability of the counting process in complex scenes, the system introduces candidate filtering based on image frame area ratio after target detection. This removes false detections of bounding boxes or backgrounds that significantly exceed reasonable size ranges, reducing the chances of large shadows, device parts, and other non-target areas being mistakenly identified as bottle caps and included in the counting. Simultaneously, the system provides an image adjustment mode for visual previewing left and right cropping of image frames during processing. Image cropping parameters are persistently saved via configuration files, allowing for the rapid construction of stable processing areas in software without adjusting camera parameters, thereby reducing the impact of image boundary regions on detection and tracking.

[0062] In terms of operation scheduling, the system adopts a processing method that combines timed triggering with asynchronous inference calls, achieving a balance between low-latency processing of continuous image streams and smooth interface display in a single-machine operating environment. The time consumption information during the inference process is displayed on the interface, and the counting results are statistically presented using a time-based sliding window statistical method and a record list summarized by minute. This allows the system to not only provide cumulative counting results but also reflect the distribution of counts over time, facilitating production line cycle time observation and alignment with external statistical reports in the time dimension.

[0063] Regarding anomaly handling during system operation, when anomalies such as stream acquisition failure or empty frames occur, the system performs protective measures on the relevant processing flows to prevent invalid images from participating in subsequent detection and counting. When improper cropping parameter settings result in invalid image frames for processing, a prompt message is provided to the operator through the human-machine interface. When the system switches between start, stop, or exit states, it explicitly releases camera resources and timing trigger-related resources to reduce the risk of anomalies caused by unreleased resources during long-term operation. Through the above modular software architecture design, the system can still output stable, interpretable, and reproducible counting results under conditions such as complex backgrounds, overlapping occlusions, and high-speed operation, while taking into account the convenience of engineering deployment, maintenance cost control, and subsequent functional expansion needs, providing a reliable data foundation for production monitoring, quality analysis, and traceability management.

[0064] The working process of the conveyor belt bottle cap counting system based on deep learning and multi-object tracking proposed in this invention is as follows: After the system starts, the camera continuously acquires image data. The image acquisition module performs pixel format conversion processing on the acquired images to obtain image frames of a unified format, and inputs the image frames to the detection and tracking module. The detection and tracking module performs bottle cap target detection on the image frames, obtains the corresponding detection boxes, and assigns a unique track ID to each detected bottle cap target using a multi-object tracking algorithm. Subsequently, the detection results enter the candidate filtering process. The system filters the detection results based on the area ratio of the detection boxes in the current image frame, eliminating abnormal detection boxes that clearly exceed the reasonable size range or false detection results generated by the background area.

[0065] After candidate filtering is completed, the system performs software cropping on the image frames during processing based on the image cropping parameters loaded in the configuration management module to eliminate the effects of black borders, occlusion, or distortion caused by image edge regions. The cropped image frames serve as input for subsequent counting processing. The counting logic module compares the historical center point of the tracked target with the virtual counting line. When it detects that the target center point crosses the virtual counting line between adjacent image frames, it triggers the corresponding counting event and updates the counting result.

[0066] During system operation, the human-machine interface module displays image frames and corresponding counting results in real time, and simultaneously shows the time consumption information and statistical charts for inference processing. Operators can adjust the cropping area through the human-machine interface module and preview the adjusted cropping effect in real time. The relevant image cropping parameters are automatically saved to the configuration file after confirmation for subsequent loading and use.

[0067] Regarding anomaly handling, when the system detects streaming failure, empty frames, or invalid image frames due to improper cropping parameter settings, it performs protective measures on the relevant processing flow and provides prompts to the operator through the human-machine interface module. Simultaneously, when the system switches between start, stop, or exit states, it explicitly releases camera resources and timing-triggered resources to ensure stable operation during continuous operation.

[0068] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A conveyor belt bottle cap counting system based on deep learning and multi-object tracking, characterized in that, The system includes: The image acquisition module is used to implement device enumeration, packet length optimization, shutdown triggering, continuous streaming, and unified conversion of multiple pixel formats through the lightweight encapsulated Hikvision GigE camera SDK, and output stable BGR image frames. The detection and tracking module is used to load a pre-trained YOLO model to detect bottle cap targets in image frames, and to assign a unique track ID to each detected target using a multi-target tracking algorithm. The counting logic module is used to set a horizontal baseline in the image frame as a virtual counting line for counting, and maintain the y-coordinate history of the center point of each target with track id. When the target center point is detected to cross the virtual counting line between adjacent image frames, a counting event is triggered, and the counting result is obtained by deduplicating the set of counted ids. The human-machine interface module is used to display image frames and counting results during the processing, and to periodically refresh the screen and trigger detection and tracking processing to display inference time information and statistical information. The configuration management module is used to persistently save image cropping parameters and the position of virtual counting lines through configuration files, and load the parameters during system startup or operation to restore the parameters.

2. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The various pixel formats include Mono8, BGR8, and BayerRG8.

3. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The human-machine interface module is used to receive image cropping parameters and virtual counting line position adjustment operations input through the human-machine interface, and to transmit the adjusted parameters to the configuration management module for saving.

4. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The human-computer interface module uses PyQt5 to build a full-screen interface.

5. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The system also includes a candidate filtering module, which filters the detection boxes in the bottle cap target detection results based on the area ratio of the current image frame after the bottle cap target detection is completed, and removes oversized detection boxes or false background detections that exceed a set threshold.

6. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The human-machine interface module is used to provide a screen adjustment mode to visually preview the image frames during processing by cropping them left and right, and to pass the obtained image cropping parameters to the configuration management module.

7. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The human-machine interface module also provides a count clearing function, which is used to perform a reset process on the counting result generated by the counting logic module and the set of counted IDs used for deduplication during the clearing operation.

8. The conveyor belt bottle cap counting system based on deep learning and multi-target tracking according to claim 1, characterized in that, The system possesses control and exception handling capabilities during operation, and is used for: When switching between start, stop, and exit states, explicitly release camera resources and timing trigger mechanisms; When a stream acquisition failure or empty frame occurs, protection measures are performed on the processes related to image acquisition and processing. When cropping parameters cause the image frame to be invalid for processing, a prompt message is provided through the human-machine interface.

9. A conveyor belt bottle cap counting method based on deep learning and multi-object tracking, characterized in that, The method includes: The lightweight Hikvision GigE camera SDK enables device enumeration, packet length optimization, shutdown triggering, continuous streaming, and unified conversion of multiple pixel formats, outputting stable BGR image frames. The pre-trained YOLO model is loaded to detect bottle cap targets in the image frame, and a multi-target tracking algorithm is used to assign a unique track ID to each detected target. A horizontal baseline is set in the image frame as a virtual counting line for counting, and the y-coordinate history of the center point of each target with track id is maintained. When the target center point is detected to cross the virtual counting line between adjacent image frames, a counting event is triggered, and the count result is obtained by deduplicating the set of counted ids. Displays image frames and counting results during the processing; The image cropping parameters and the position of the virtual counter line are persistently saved through the configuration file, and the parameters are loaded during system startup or operation to restore the parameters.

Images