Tobacco shred detection method and system based on vision

Abstract

The invention discloses a vision-based cut tobacco detection method and system, and the method comprises the steps: carrying out the asynchronous parallel processing operation of video stream decoupling and video stream reasoning on a to-be-detected video stream collected by an image collection unit through introducing a producer and consumer model; in other words, the first calculation unit synchronously realizes the video stream reasoning operation of the second calculation unit in the video stream decoupling process, and when the first calculation unit carries out first preprocessing on the to-be-detected image, the second calculation unit is synchronously configured to carry out data calculation on the initial image according to the calculation task; and the first calculation unit draws the tobacco shred image according to the calculation result, and judges the actual tobacco shred image and the reference tobacco shred image to obtain a detection result. The utilization rate of the first calculation unit and the second calculation unit is improved, the post-processing process is parallelized through a parallel result drawing algorithm, the time for waiting for result return in sequential processing is shortened, and the whole detection logic is smoother.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a vision-based tobacco detection method and system. Background Art

[0002] Tobacco detection can monitor and control the quality of tobacco in the production process to improve the quality and characteristics of subsequent tobacco products. The existing tobacco detection uses a camera to collect video data of cut tobacco, and then inputs the video data into the GPU for image processing and parameter calculation. During this process, the CPU is in a state of waiting for the GPU to return the result. After the GPU completes the calculation, it also needs to wait for the CPU to re-issue the calculation task before performing image processing and parameter calculation again. This synchronous execution calculation method leads to a waste of computing resources of hardware such as the CPU and GPU, and is the least efficient method. In addition, when the detection accuracy is high, the video stream resolution is high, such as 8k resolution. At this time, the image size of its raw format file is 100M, and the amount of data transmitted by the camera is large. When in computer hardware, from the cache to the computer register, the data transmission delay caused by its huge amount of data is also high. If computing hardware such as GPU is used, the calculation delay will be higher at this time. Therefore, if the computer directly reads the video stream from the camera for detection, it will be affected by the hardware data transmission speed, the processing speed of the tobacco detection algorithm, etc., and the result is that the video stream of the detection interface appears stuck, frame loss, etc. Summary of the invention

[0003] In view of this, an object of the present invention is to provide a vision-based tobacco detection method and system.

[0004] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0005] In a first aspect, the present invention provides a vision-based tobacco detection method, applicable to a first computing unit, the method comprising:

[0006] Acquire the video stream to be detected containing tobacco collected by the image acquisition unit, and input the video stream to be detected into the producer-consumer model, wherein the producer-consumer model includes a producer thread and a consumer thread;

[0007] The producer thread decouples the video stream to be detected and generates multiple video frames to be detected, and inputs the multiple video frames to be detected into the cache queue one by one;

[0008] The consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning to obtain a plurality of to-be-detected images, and the consumer thread and the producer thread process in parallel;

[0009] Performing a first preprocessing on the image to be detected to obtain an initial image;

[0010] Obtaining a computing task, controlling the second computing unit to perform data computing on the initial image according to the computing task, outputting a plurality of computing results and returning them;

[0011] Multiple tobacco images are drawn in parallel according to multiple calculation results to obtain actual tobacco images, and the actual tobacco images are compared with reference tobacco images to obtain detection results.

[0012] In some embodiments, the first computing unit is a CPU and the second computing unit is a GPU.

[0013] In some embodiments, the consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning, further comprising:

[0014] Performing a second preprocessing on the video frame to be detected;

[0015] Inputting the second preprocessed video frame to be detected into the deep learning model to perform target detection, classification and tracking processing to obtain an inference result;

[0016] Screen, filter, target track and visualize the inference results;

[0017] An initial image corresponding to the video frame to be detected is obtained.

[0018] In some embodiments, the second pre-processing includes at least one of resizing, color space conversion, and image enhancement.

[0019] In some embodiments, performing a first preprocessing on the image to be detected further includes:

[0020] Obtain basic image size information of the image to be detected, and match the basic image size information with preset image size information;

[0021] If the value of the basic image size is smaller than the value of the preset image size, the image to be detected is padded so that the image size of the padded image to be detected is consistent with the preset image size;

[0022] If the basic image size is larger than the preset image size, the image to be detected is segmented in a preset manner according to the preset image size to generate multiple segmented images, and the multiple segmented images are input into the second calculation unit according to the segmentation order.

[0023] In some embodiments, the image to be detected before segmentation is recorded as an original image, and each segmented image has an original position information, and the original position information is the position data of the segmented image in the original image;

[0024] The parallel drawing of multiple tobacco images according to multiple calculation results also includes:

[0025] The original position information is obtained, and the calculation results of the segmented images are stitched according to the original position information to generate an actual tobacco image corresponding to the original image.

[0026] In some embodiments, the computing task includes a first thread computing task and a second thread computing task in the same computing cycle;

[0027] According to the calculation task, the second calculation unit is controlled to perform data calculation on the initial image, and multiple calculation results are output and returned including:

[0028] The second computing unit performs data computing on the initial image according to the computing task of the first thread, wherein the data computing includes at least one of tobacco length computing and tobacco width computing, and the computing result includes an actual tobacco length value and / or an actual tobacco width value;

[0029] Drawing multiple tobacco images in parallel according to multiple calculation results to obtain an actual tobacco image also includes:

[0030] The first computing unit draws the tobacco image based on the calculation result of the initial image in the previous operation cycle according to the second thread computing task, and the second thread computing task is processed in parallel with the first thread computing task.

[0031] In some embodiments, the second computing unit performs data calculation on the initial image according to the first thread computing task, including:

[0032] Input the initial image into the CUDA model for data calculation to obtain initial calculation result data;

[0033] The initial calculation result data is subjected to non-maximum suppression processing to obtain the calculation result.

[0034] In some embodiments, the method further comprises:

[0035] An adjustment instruction is generated according to the detection result, and the adjustment instruction is sent to the tobacco cutting device to control the tobacco cutting device to adjust the tobacco cutting parameters.

[0036] In a second aspect, the present invention further provides a vision-based tobacco detection system, comprising a tobacco cutting device, a conveyor belt, a light source assembly, an image acquisition unit and a control unit, wherein the tobacco cutting device is used to cut tobacco; the conveyor belt is arranged at the output end of the tobacco cutting device, and the conveyor belt is used to transport the cut tobacco; the light source assembly comprises a first light source and a second light source, and the first light source and the second light source are arranged at intervals along the conveying direction of the conveyor belt; the image acquisition unit is arranged above the conveyor belt and between the first light source and the second light source, and the image acquisition unit is used to acquire a video stream to be detected containing tobacco; the control unit is electrically connected to the image acquisition unit and the tobacco cutting device, and the control unit is used to execute the tobacco detection method described in the first aspect.

[0037] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0038] The above technical solution introduces the producer-consumer model to perform asynchronous parallel processing operations of video stream decoupling and video stream reasoning on the video stream to be detected collected by the image acquisition unit, that is, the first computing unit synchronously implements the video stream reasoning operation of the second computing unit during the video stream decoupling process, and when the first computing unit performs the first preprocessing on the image to be detected, the second computing unit is synchronously configured to perform data calculation on the initial image according to the calculation task, and the first computing unit then draws the tobacco image according to the calculation result, and judges the actual tobacco image with the reference tobacco image to obtain the detection result. In this process, asynchronous heterogeneous computing of the first computing unit and the second computing unit is realized, the utilization rate of the first computing unit and the second computing unit is improved, and the post-processing process is parallelized through the parallel result drawing algorithm, which reduces the time waiting for the result to be returned in the sequential processing, making the entire detection logic smoother. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a step diagram of the vision-based tobacco detection method;

[0041] Figure 2 is a first flow chart of a vision-based tobacco detection method;

[0042] Figure 3 is a second flow chart of a tobacco detection method based on vision;

[0043] Figure 4 It is the logical processing flow chart of the producer-consumer model;

[0044] Figure 5 is a flow chart for performing a first preprocessing on the image to be detected;

[0045] Figure 6 It is a flow chart of heterogeneous asynchronous configuration of CPU and GPU computing tasks;

[0046] Figure 7 It is a flowchart of tobacco image drawing;

[0047] Figure 8 It is a before and after comparison diagram of the utilization rate of the first computing unit using the vision-based tobacco detection method;

[0048] Fig. 9 It is a before and after comparison diagram of the utilization rate of the second computing unit using the vision-based tobacco detection method;

[0049] Fig.10 It is a schematic diagram of a vision-based tobacco detection system.

[0050] Reference numerals:

[0051] 1. Conveyor belt;

[0052] 2. The first light source;

[0053] 3. Second light source;

[0054] 4. Image acquisition unit. DETAILED DESCRIPTION

[0055] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0056] See also Figures 1 to 9 In a first aspect, this embodiment provides a vision-based tobacco detection method, applicable to a first computing unit, the method comprising:

[0057] S1. Obtain a video stream to be detected containing tobacco collected by an image acquisition unit, and input the video stream to be detected into a producer-consumer model, where the producer-consumer model includes a producer thread and a consumer thread;

[0058] S2, the producer thread decouples the video stream to be detected and generates multiple video frames to be detected, and inputs the multiple video frames to be detected into the cache queue one by one;

[0059] S3, the consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning to obtain a plurality of to-be-detected images, and the consumer thread and the producer thread process in parallel;

[0060] S4, performing a first preprocessing on the image to be detected to obtain an initial image;

[0061] S5, obtaining a computing task, controlling the second computing unit to perform data computing on the initial image according to the computing task, outputting a plurality of computing results and returning them;

[0062] S6. Drawing multiple tobacco shred images in parallel according to multiple calculation results to obtain actual tobacco shred images, and comparing the actual tobacco shred images with reference tobacco shred images to obtain detection results.

[0063] In step S1, the image acquisition unit can acquire the video stream to be detected containing tobacco shreds. Specifically, the definition of the video stream to be detected can be set according to actual needs. In this embodiment, the video stream to be detected is input into the producer-consumer model.

[0064] Please refer to Figure 4 As shown in the figure, the producer-consumer model includes producer threads and consumer threads. The producer-consumer model is a common concurrent programming model used to solve the synchronization and coordination problems of sharing limited resources between multiple producer threads and consumer threads. In this model, the producer thread is responsible for producing data or tasks, and the consumer thread is responsible for consuming these data or tasks. The producer-consumer model usually involves a shared buffer, that is, a cache queue. The producer thread puts the data or tasks into the buffer, and the consumer thread takes out and processes the data or tasks from the cache queue. The producer thread and the consumer thread communicate and collaborate through the cache queue.

[0065] In step S2, the producer thread can decouple the video stream to be detected and generate multiple video frames to be detected, and input the multiple video frames to be detected into the cache queue one by one. The decoupling of the video stream refers to splitting the video stream into independent video frames for processing, rather than processing the entire video stream as a continuous data stream; this decoupling can improve parallelism and processing efficiency, so that each frame can be processed independently, thereby achieving higher real-time performance.

[0066] See also Figure 4, the consumer thread can read a preset number of video frames to be detected from the cache queue, and perform video stream reasoning on multiple video frames to be detected to obtain multiple images to be detected. In this process, the producer thread and the consumer thread can be processed in parallel to improve the data processing speed of the video stream to be detected. Further, this embodiment uses a second computing unit to perform video stream reasoning, wherein the video stream reasoning process can be performed by a deep learning model in the second computing unit, and the preset number can be set according to actual needs. Preferably, the preset number is set according to the batch of tobacco shreds and the optimal throughput of the second computing unit, that is, there are multiple video frames to be detected decoupled from the same batch of tobacco shreds, and the second computing unit performs video stream reasoning by selecting a preset number of video frames to be detected. In this process, the preset number depends on the optimal throughput of the second computing unit. For example, when the optimal throughput of the second computing unit is 10 video frames to be detected, and the video frames to be detected corresponding to the same batch of tobacco are 12, 10 of them are selected and input into the second computing unit for video stream reasoning.

[0067] In step S4, the image to be detected needs to be preprocessed to obtain an initial image. The purpose of this step is to regularize the images to be detected of different resolutions into images to be detected of the same resolution, thereby improving the adaptability of the entire detection method to video streams to be detected of different resolutions.

[0068] In step S5, the first computing unit issues a computing task to the second computing unit. The second computing unit performs data calculation on the initial image according to the computing task, outputs multiple computing results and returns them. Specifically, the computing task includes the calculation of morphological parameters such as the length and width of the tobacco. The algorithm model corresponding to the computing task can be pre-configured in the second computing unit to improve processing efficiency.

[0069] In step S6, the first calculation unit draws multiple tobacco images in parallel according to the multiple calculation results obtained to obtain an actual tobacco image, and compares the actual tobacco image with the reference tobacco image to obtain a detection result. The actual tobacco image is an image of the tobacco after actual cutting obtained by drawing the video stream to be detected, which contains the size information of the tobacco, the quality information of the tobacco, etc. This embodiment is mainly used to monitor the size information of the tobacco. Therefore, when comparing the actual tobacco image with the reference tobacco image, the actual size information of the tobacco and the reference size information of the tobacco of the reference tobacco image are judged to obtain a detection result. The detection result may include the difference between the actual size information of the tobacco and the reference size information.

[0070] Furthermore, in some embodiments, the method further comprises:

[0071] An adjustment instruction is generated according to the detection result, and the adjustment instruction is sent to the tobacco cutting device to control the tobacco cutting device to adjust the tobacco cutting parameters.

[0072] The adjustment instruction includes the adjustment parameters of the blade in the tobacco cutting device. For example, when the length difference between the actual size information of the tobacco shown in the detection result and the reference size information is positive 5mm, it means that the length of the tobacco in the actual size information is too long, and the adjustment parameters of the blade for cutting the length of the tobacco need to be adjusted, including the displacement deviation information of the blade, etc., to reduce the length difference value to within the allowable error range. It should be noted that the tobacco cutting device can be a commercially available tobacco cutting device, and the specific structure of the tobacco cutting device is not described in detail in this embodiment.

[0073] It should also be noted that the second computing unit and the first computing unit can also use asynchronous heterogeneous computing to realize the calculation of initial images of multiple batches of tobacco and the drawing of multiple batches of tobacco images. That is, according to the batch distinction of tobacco images, step S5 calculates the calculation task of the initial image corresponding to the video stream to be detected of the current batch of tobacco, and step S6 calculates the tobacco image drawing of the calculation result of the initial image corresponding to the video stream to be detected of the previous batch of tobacco. Under this logical idea, step S5 and step S6 can be processed in parallel to make full use of the first computing unit and the second computing unit.

[0074] In some embodiments, the first computing unit is a CPU, and the second computing unit is a GPU. GPU is the abbreviation of graphics processing unit, which is a processor specially used for graphics rendering and calculation. Unlike the central processing unit (CPU) which is mainly responsible for general-purpose calculations, the design and architecture of GPU are more focused on graphics-related computing tasks. In this embodiment, the relevant image processing of the video stream to be detected containing tobacco can be implemented by GPU, while the image preprocessing or computing task allocation in the video stream to be detected can be implemented by CPU. Through the collaborative operation of GPU and CPU, the utilization rate of CPU and GPU is improved, and at the same time, the overall image data processing efficiency of tobacco detection is improved, thereby improving the efficiency of tobacco detection.

[0075] This embodiment introduces a producer-consumer model to perform asynchronous parallel processing operations of video stream decoupling and video stream reasoning on the video stream to be detected collected by the image acquisition unit, that is, the first computing unit synchronously implements the video stream reasoning operation of the second computing unit during the video stream decoupling process, and when the first computing unit performs the first preprocessing on the image to be detected, the second computing unit is synchronously configured to perform data calculation on the initial image according to the calculation task, and the first computing unit then draws the tobacco image according to the calculation result, and judges the actual tobacco image with the reference tobacco image to obtain the detection result. In this process, asynchronous heterogeneous computing of the first computing unit and the second computing unit is realized, the utilization rate of the first computing unit and the second computing unit is improved, and the post-processing process is parallelized through the parallel result drawing algorithm, which reduces the time waiting for the result to be returned in the sequential processing, making the entire detection logic smoother.

[0076] See also Fig.10 Corresponding to the aforementioned method, in a second aspect, the present embodiment further provides a vision-based tobacco detection system, comprising a tobacco cutting device, a conveyor belt 1, a light source assembly, an image acquisition unit 4 and a control unit, wherein the tobacco cutting device is used to cut tobacco; the conveyor belt 1 is arranged at the output end of the tobacco cutting device, and the conveyor belt 1 is used to transport the cut tobacco; the light source assembly comprises a first light source 2 and a second light source 3, and the first light source 2 and the second light source 3 are arranged at intervals along the conveying direction of the conveyor belt 1; the image acquisition unit 4 is arranged above the conveyor belt 1 and between the first light source 2 and the second light source 3, and the image acquisition unit 4 is used to acquire a video stream to be detected containing tobacco; the control unit is electrically connected to the image acquisition unit 4 and the tobacco cutting device, and the control unit is used to execute the tobacco detection method described in the first aspect.

[0077] Optionally, the first light source 2 and the second light source 3 may be linear array LED light sources, the image acquisition unit 4 may be a CCD camera, and the aforementioned tobacco detection method may be implemented by the tobacco detection system shown in this embodiment.

[0078] In some embodiments, the consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning, further comprising:

[0079] Performing a second preprocessing on the video frame to be detected;

[0080] Inputting the second preprocessed video frame to be detected into the deep learning model to perform target detection, classification and tracking processing to obtain an inference result;

[0081] Screen, filter, target track and visualize the inference results;

[0082] An initial image corresponding to the video frame to be detected is obtained.

[0083] In some embodiments, the second pre-processing includes at least one of resizing, color space conversion, and image enhancement.

[0084] This embodiment improves the recognition degree of the tobacco part in the video frame to be detected by resizing, spatial color conversion and image enhancement of the video frame to be detected, and realizes target detection, classification and tracking processing with tobacco as the target through deep learning to obtain inference results, and then screens and filters the inference results for target tracking and result visualization to obtain an initial image corresponding to the video frame to be detected.

[0085] See also Figure 5 In some embodiments, performing a first preprocessing on the image to be detected further includes:

[0086] Obtain basic image size information of the image to be detected, and match the basic image size information with preset image size information;

[0087] If the value of the basic image size is smaller than the value of the preset image size, the image to be detected is padded so that the image size of the padded image to be detected is consistent with the preset image size;

[0088] If the basic image size is larger than the preset image size, the image to be detected is segmented in a preset manner according to the preset image size to generate multiple segmented images, and the multiple segmented images are input into the second calculation unit according to the segmentation order.

[0089] In this embodiment, the basic image size is matched with the preset image size. Specifically, the basic image size includes the length value and width value of the current image, and the preset image size includes the preset image length value and image width value, which can also be understood as the resolution information of the preset image. The length value of the image to be detected is compared with the preset image length value to determine its size, and the width value of the image to be detected is compared with the preset image width value to determine its size.

[0090] If the size of the basic image is smaller than the preset image size, that is, the resolution of the image to be detected is smaller than the resolution of the preset image, the image to be detected is padded to the size of the preset image.

[0091] If the basic image size is larger than the preset image size, that is, the resolution of the image to be detected is larger than the resolution of the preset image, the image to be detected is segmented in a preset manner. The specific preset method can be segmentation according to a preset overlap ratio to obtain multiple segmented images, and the image size of each segmented image is consistent with the preset image size during segmentation. The multiple segmented images are organized into an image queue, and each segmented image in the queue is a part of the image to be detected. This step can avoid the problem of over-scaling of the image to be detected, and can detect different resolutions.

[0092] In some preferred embodiments, the preset image size is a standard size, that is, 1280*1280.

[0093] In some embodiments, the image to be detected before segmentation is recorded as an original image, and each segmented image has an original position information, and the original position information is the position data of the segmented image in the original image;

[0094] The parallel drawing of multiple tobacco images according to multiple calculation results also includes:

[0095] The original position information is obtained, and the calculation results of the segmented images are stitched according to the original position information to generate an actual tobacco image corresponding to the original image.

[0096] This embodiment shows the drawing process of the tobacco cut image of the first computing unit for multiple segmented images in the aforementioned image queue. Specifically, each segmented image has an original position information, which is the position data of the segmented image in the original image. In this embodiment, the second computing unit can set multiple threads to perform separate calculations on multiple segmented images at the same time, without having to calculate the segmented images one by one in the segmentation order, which can further improve the calculation efficiency of the second computing unit.

[0097] One sliced ​​image corresponds to one calculation result. If there are multiple calculation results in the same graphics queue, you can Figure 7 As shown, the first computing unit can synchronously receive the calculation results returned by multiple second computing units. At this time, the calculation results are out of order, but each calculation result corresponds to an original position information stored and mapped therewith. The calculation results of the segmented image are drawn and spliced ​​according to the original position information to generate an actual tobacco image corresponding to the original image.

[0098] Specifically, the logic for generating the original position information may be: generating the original position information based on the coordinates of the upper left corner of each segmented image relative to the position of the original image.

[0099] In some optional embodiments, there are multiple images to be detected in the same batch, and the computing tasks corresponding to the images to be detected can also be opened in the second computing unit to open multiple threads for parallel processing to improve the processing speed of the images to be detected of the same batch of tobacco. The images to be detected in this batch can also have the reference position information required for drawing. It should be noted that the reference position information here is different from the original position information of the segmented image. The reference position information is the timestamp information of the current image to be detected in the video stream to be detected. The first computing unit can also add multiple threads to draw tobacco images for each image to be detected, and after obtaining the actual tobacco image, the multiple actual tobacco images are arranged and integrated according to the reference position information to obtain a set of actual tobacco images of the current batch, which is convenient for subsequent tobacco detection and data regularization and traceability.

[0100] See also Figure 6 , in some embodiments, the computing task includes a first thread computing task and a second thread computing task in the same computing cycle;

[0101] According to the calculation task, the second calculation unit is controlled to perform data calculation on the initial image, and multiple calculation results are output and returned including:

[0102] The second computing unit performs data computing on the initial image according to the computing task of the first thread, wherein the data computing includes at least one of tobacco length computing and tobacco width computing, and the computing result includes an actual tobacco length value and / or an actual tobacco width value;

[0103] Drawing multiple tobacco images in parallel according to multiple calculation results to obtain an actual tobacco image also includes:

[0104] The first computing unit draws the tobacco image based on the calculation result of the initial image in the previous operation cycle according to the second thread computing task, and the second thread computing task is processed in parallel with the first thread computing task.

[0105] In this embodiment, the first thread computing task and the second thread computing task are independent of each other. The first thread computing task can be understood as the computing task that the second computing unit needs to execute, and the second thread computing task can be understood as the computing task that the first computing unit needs to execute.

[0106] The concept of operation cycle is introduced in this embodiment. The operation cycle represents the cycle required for a single tobacco detection method. Furthermore, the operation cycle is associated with a batch of tobacco, and the current operation cycle represents the time required to execute a batch of tobacco detection detection methods. In this embodiment, the first thread computing task may correspond to the tobacco image drawing step of the initial image in the previous operation cycle, and the second thread computing task corresponds to the data calculation process of the initial image in the current operation cycle. The two are independent of each other and processed in parallel, avoiding the problem that the calculation steps of the first computing unit must rely on the calculation results of the second computing unit and can only wait in situ for the second computing unit to perform data calculations, thereby improving the utilization of the first computing unit and the second computing unit. When the first computing unit is a CPU and the second computing unit is a GPU, this method can significantly improve the utilization of the two. Figure 8 and Fig. 9 shown.

[0107] In some embodiments, the second computing unit performs data calculation on the initial image according to the first thread computing task, including:

[0108] Input the initial image into the CUDA model for data calculation to obtain initial calculation result data;

[0109] The initial calculation result data is subjected to non-maximum suppression processing to obtain the calculation result.

[0110] In this embodiment, a CUDA model is configured on the second computing unit, and the data calculation function of the initial image is implemented through the CUDA model to reduce the occupancy rate of the first computing unit. On this basis, the first computing unit can synchronously perform the drawing operation of the actual tobacco image.

[0111] In addition, this embodiment introduces a non-maximum suppression algorithm to select the most representative target from a set of overlapping candidate boxes or bounding boxes and suppress other overlapping bounding boxes, that is, to perform border filtering on a single initial image, thereby improving the accuracy and stability of the detection results and reducing false detections and redundancy caused by overlapping bounding boxes.

[0112] The above embodiment realizes a high-throughput tobacco detection method by integrating an advanced tobacco detection model and a traditional computer vision algorithm optimized by CUDA. This method effectively decouples the reading operation of the video stream from the computing task, allowing the two processes to be executed in parallel, thereby significantly improving the processing efficiency. Specifically, the CPU utilization rate increased by about 11%, while the GPU utilization rate increased by 2%, which greatly reduced the idle period of the GPU when waiting for computing tasks. Through this optimization, the problems of slow processing speed of tobacco video stream and video frame loss are solved, and the smoothness and real-time performance of video stream processing are significantly enhanced.

[0113] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0114] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0115] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A tobacco detection method based on vision, characterized in that: Applicable to a first computing unit, the method comprises: Acquire the video stream to be detected containing tobacco collected by the image acquisition unit, and input the video stream to be detected into a producer-consumer model, wherein the producer-consumer model includes a producer thread and a consumer thread; The producer thread decouples the video stream to be detected and generates a plurality of video frames to be detected, and inputs the plurality of video frames to be detected into a cache queue one by one; The consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning to obtain a plurality of to-be-detected images, and the consumer thread and the producer thread are processed in parallel; Performing a first preprocessing on the image to be detected to obtain an initial image; Obtaining a computing task, and controlling the second computing unit to perform data computing on the initial image according to the computing task, outputting a plurality of computing results and returning them; A plurality of tobacco shred images are drawn in parallel according to the plurality of calculation results to obtain an actual tobacco shred image, and the actual tobacco shred image is compared with a reference tobacco shred image to obtain a detection result.

2. The visual-based tobacco detection method according to claim 1, characterized in that: The first computing unit is a CPU, and the second computing unit is a GPU.

3. The visual-based tobacco detection method according to claim 1 or 2, characterized in that: The consumer thread extracts a preset number of to-be-detected video frames from the cache queue and inputs them into the second computing unit for video stream reasoning, and further includes: Performing a second preprocessing on the video frame to be detected; Inputting the video frame to be detected after the second preprocessing into the deep learning model to perform target detection, classification and tracking processing to obtain an inference result; Screening, filtering, target tracking and visualization of the inference results; An initial image corresponding to the video frame to be detected is obtained.

4. The visual-based tobacco detection method according to claim 3, characterized in that: The second preprocessing includes at least one of size adjustment, color space conversion and image enhancement.

5. The visual-based tobacco detection method according to claim 4, characterized in that: Performing a first preprocessing on the image to be detected further includes: Acquire basic image size information of the image to be detected, and match the basic image size information with preset image size information; If the value of the basic image size is smaller than the value of the preset image size, padding the image to be detected so that the image size of the padded image to be detected is consistent with the preset image size; If the basic image size is larger than the preset image size, the image to be detected is segmented in a preset manner according to the preset image size to generate multiple segmented images, and the multiple segmented images are input into the second calculation unit according to the segmentation order.

6. The visual-based tobacco detection method according to claim 5, characterized in that: The image to be detected before segmentation is recorded as an original image, each of the segmented images has an original position information, and the original position information is the position data of the segmented image in the original image; The step of performing parallel drawing of a plurality of tobacco images according to a plurality of the calculation results further comprises: The original position information is obtained, and the calculation results of the segmented images are drawn and combined one by one according to the original position information to generate an actual tobacco image corresponding to the original image.

7. The visual-based tobacco detection method according to claim 1, characterized in that: The computing tasks include a first thread computing task and a second thread computing task in the same computing cycle; Controlling the second computing unit to perform data calculation on the initial image according to the computing task, outputting a plurality of computing results and returning the results includes: The second computing unit performs data calculation on the initial image according to the first thread computing task, the data calculation includes at least one of tobacco length calculation and tobacco width calculation, and the calculation result includes an actual tobacco length value and / or an actual tobacco width value; Drawing a plurality of tobacco images in parallel according to the plurality of calculation results to obtain an actual tobacco image further comprises: The first computing unit draws the tobacco image based on the calculation result of the initial image in the previous operation cycle according to the second thread computing task, and the second thread computing task is processed in parallel with the first thread computing task.

8. The visual-based tobacco detection method according to claim 7, characterized in that: The second computing unit performing data calculation on the initial image according to the first thread computing task includes: Inputting the initial image into the CUDA model for data calculation to obtain initial calculation result data; The initial calculation result data is subjected to non-maximum suppression processing to obtain the calculation result.

9. The visual-based tobacco detection method according to claim 1, characterized in that: The method further comprises: An adjustment instruction is generated according to the detection result, and the adjustment instruction is sent to the tobacco cutting device to control the tobacco cutting device to adjust the tobacco cutting parameters.

10. A vision-based tobacco detection system, characterized in that: include: Tobacco cutting equipment, used for cutting tobacco; A conveyor belt, arranged at the output end of the tobacco cutting device, and used to transport the cut tobacco; A light source assembly, comprising a first light source and a second light source, wherein the first light source and the second light source are arranged at intervals along a conveying direction of the conveyor belt; An image acquisition unit is arranged above the conveyor belt and between the first light source and the second light source, and is used for acquiring a video stream to be detected containing tobacco shreds; A control unit is electrically connected to the image acquisition unit and the tobacco cutting device, and the control unit is used to execute the tobacco detection method according to any one of claims 1 to 9.

Images