A feature recognition method for finished cigarette boxes based on machine vision

Abstract

This invention discloses a machine vision-based method for identifying features in finished cigarette boxes, belonging to the field of intelligent logistics technology. Using an intelligent camera, the method identifies the presence of physical objects in the warehouse, whether they are wrapped, the specifications, whether the boxes are recycled, the number of boxes, and whether they are in-process or finished products. This method then compares this information with inventory information to complete inventory checks. This method offers excellent accuracy and stability, addressing the safety risks associated with manual inventory checks. It also improves inventory efficiency, saves human resources, and enables unmanned inventory counting. It also saves real-time images of cigarettes being counted to verify the authenticity and effectiveness of physical inventory checks.

Description

Technical Field

[0001] The present invention relates to the field of intelligent logistics technology, and more specifically, to a method for identifying features of finished cigarette boxes based on machine vision. Background Art

[0002] Finished cigarettes produced by the Tobacco Group's cigarette factory are stacked on pallets in boxes according to prescribed standards and stored in high-bay warehouses. Inventory counts rely on manual vision, which poses safety risks associated with high-altitude operations and results in low efficiency, accuracy, and timeliness. Advances in image recognition and automation technologies have led to a solution for automated inventory counting based on machine vision. Intelligent cameras capture the characteristics of each pallet of cigarette cases in the warehouse. Algorithmic software analyzes the images to identify the physical location of the case, whether it is wrapped, the specification, whether the case is recycled, the number of cases in the stack, and whether the case is in-process or finished. This information is then compared with inventory information to complete the inventory count. Human intervention is unnecessary during the count, as the stacker crane operates continuously. The visual recognition system also records images of each pallet count, enabling pallet traceability and manual corrections.

[0003] Perform feature analysis on different recognition requirements of finished cigarette boxes. For each recognition requirement, select one or more features to make the recognition algorithm, compare the features with a large number of photographs, and adjust the feature selection for different recognition requirements based on the recognition algorithm's recognition results on a large number of pictures.

[0004] High bay warehouse environment:

[0005] (1) The storage locations of finished cigarette products in the high-bay warehouse are defined by row-column-layer positions. The space size of each storage location is uniform. After the cigarette boxes are stacked on the pallet and put into the warehouse, there is very little remaining space, which limits the installation location of the camera.

[0006] (2) The background of the empty cargo space is more complex, and the open ambient light has a more obvious impact, causing great interference;

[0007] (3) There are many types of cigarette brands, and the sub-brands have little difference in appearance, which makes them easy to confuse;

[0008] (4) Some solid pallet cigarette stacks are wrapped with film to prevent cigarette pieces from falling (including various cigarette stack types), which will cover the characteristics of the cigarette box;

[0009] (5) The placement of the cigarettes after assembly is not uniform. Some have the front side (Chinese characters) in front, some have the back side (letters) in front, and some are upside down (distinguished by the front and back of the pattern). Multiple models of the same brand characteristics should be built.

[0010] To achieve accurate identification and statistics, including the presence or absence of a shelf, whether it is wrapped, product specifications, pallet type, presence or absence of a barcode, recycling bin label, and number of cigarettes, the following must be considered:

[0011] (1) How to accurately and quickly (in real time) identify targets among stacks of various brands of cigarette boxes?

[0012] (2) How to effectively construct and organize a reliable recognition algorithm and implement it smoothly;

[0013] (3) Real-time performance is an important issue that is difficult to resolve. Image processing speed and algorithm operation speed are one of the main bottlenecks affecting the real-time performance of the visual system;

[0014] (4) Stability is the first issue to be considered in the visual system. Regardless of whether it is a position-based, image-based or hybrid visual servoing method, it faces the following problems: When the difference between image features is very small, in order to ensure the stability of the system, how to consider combining global features with local features, that is, enhancing the feature area and ensuring global convergence; in order to avoid the failure of a single algorithm to recognize, it is necessary to consider how to use other supplementary algorithms to ensure stability and accuracy. Summary of the Invention

[0015] The purpose of this invention is to solve the above problems, realize machine vision inventory, and provide a method for identifying features of finished cigarette boxes based on machine vision. The following technical solutions are adopted:

[0016] In order to achieve the above-mentioned purpose, the present invention is implemented by adopting the following technical solutions: the method described includes a method for identifying whether there are physical objects in the cargo space; a method for identifying whether the cigarette stack is wrapped with film; a method for identifying the specifications; a method for identifying whether the cigarette boxes are recycled; a method for identifying the number of stack-type cigarette boxes; and a method for identifying work-in-progress / finished products.

[0017] Preferably, the inventory cigarette box features required for the method include: the presence of cigarette stacks in the cargo space, inventory information of empty cargo spaces, inventory information of pallet groups, reflected light spots after the cigarette stacks are wrapped with film, characters, patterns, colors, and barcodes on the cigarette boxes, white labels affixed to recycled cigarette boxes, marks on fixed positions, main views and top views of the cigarette box stack type, and the cigarette box No. 1 project barcode.

[0018] Preferably, the method for identifying whether there are physical objects in the cargo location is: extract the colors of various cigarette boxes when there are cigarette stacks in the cargo location to establish a model, and then search for the color corresponding to the model in the region of interest (ROI). If the color is found, it means there is a cigarette stack in the cargo location, otherwise it is an empty cargo location. The images of the empty cargo location and the pallet group have a greater impact on the effect of identifying whether there are physical objects in the cargo location, and the inventory information of the empty cargo location and the pallet group is required to improve the recognition accuracy.

[0019] Preferably, the method for identifying whether the cigarette stack is wrapped with film is as follows: after the cigarette stack is wrapped with film, the collected image will have a very bright light spot under the lighting angle. Based on this feature, a grayscale histogram analysis algorithm is used to convert the color image into a grayscale image point by point in the area of interest, and then the grayscale histogram structure data of this area is calculated and analyzed, and the contrast in the structure data is used to determine whether it is wrapped with film.

[0020] Preferably, the method for identifying the product specification is:

[0021] Step (1) Pattern training, the process is: select a training pattern, set the training area and origin, set the training parameters, train the pattern, evaluate the trained features, the pattern model training principles are: select a representative pattern with consistent features; reduce unnecessary features and image noise; only train important features;

[0022] Step (2) Runtime algorithm: set runtime parameters, define the search area, obtain the runtime pattern, run the smart camera algorithm, and obtain the results.

[0023] Preferably, the method for identifying whether a cigarette box is recycled is as follows: during the production process, the recycling box used is marked by a white label and a number written in a fixed position;

[0024] Step (1) uses a pattern matching algorithm to first establish a white label pattern model and search for matching targets in the image. If the value is 1, it means there is a label, which is a recycled cigarette box; if the value is 0, it means there is no label, which is a new cigarette box;

[0025] Step (2) For the method of writing numbers at a fixed position, an OCR character recognition algorithm is used to read the fixed ROI. The number read out is the number of times the cigarette box has been recycled.

[0026] Preferably, the method for identifying the number of stack-type cigarette boxes is:

[0027] Step (1) collecting images of each stack type and each position model and performing image processing;

[0028] Step (2) The main view model (full stack, each layer) and the top view model (each layer, each position) are established;

[0029] Step (3) real-time image processing of cargo locations;

[0030] Step (4) determines whether the main view is full of stacks. If so, obtain the corresponding stack type and total quantity according to the main view full stack model, otherwise proceed to the next step;

[0031] Step (5) determines whether the main view has four layers. If so, count the number of the fourth layer + the number of the first three layers based on the top view. Otherwise, proceed to the next step.

[0032] Step (6) determines whether the main view has three layers. If so, count the number of the third layer + the number of the first two layers based on the top view. Otherwise, proceed to the next step.

[0033] Step (7) determines whether the main view is the second layer. If so, count the number of the second layer + the number of the previous layer based on the top view, otherwise proceed to the next step;

[0034] Step (8) determines whether the main view is one layer. If so, the number of the first layer is counted based on the top view. Otherwise, it is determined to be an empty storage location or pallet group.

[0035] Preferably, the WIP / finished product identification method is based on the fact that WIP products do not have a unique engineering code, while finished products have a unique engineering code. The WIP / finished product identification algorithm first creates a WIP pattern model, locates the target location within the image, then reads the barcode at that location. Multiple results are ORed together. If any result is 1, the product is a finished product; if any result is 0, the product is WIP.

[0036] The present invention has the following beneficial effects: It can identify the presence of finished cigarettes in the warehouse, including the presence of physical objects in the warehouse, whether they are wrapped, the specification, whether the cigarette boxes are recycled, the number of cigarette boxes in stacks, and whether they are in-process or finished products. This system has high accuracy and stability, eliminating the safety risks of manual inventory counting. It also improves inventory counting efficiency, saves human resources, and enables unmanned inventory counting. It also saves real-time images of cigarette inventory counting to verify the authenticity and effectiveness of physical inventory counting. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagrams of the pattern characteristics of several finished cigarette boxes of the present invention;

[0038] Figure 2 Schematic diagram of several types of material stacks stored in the high-bay warehouse for finished cigarette products of the present invention;

[0039] Figure 3 FIG2 is an example of a front view and a top view of a 28-piece stack of standard cigarette boxes of the present invention;

[0040] Figure 4 FIG. 2 is a flow chart of the number identification of stack-type cigarette boxes of the present invention.

[0041] Among them, 1. The location of the specification pattern on the cigarette box; 2. The location of the white label on the recycled cigarette box; 3. The location of the No. 1 project barcode; 4. The location of the recycling number mark; 5. Sealing tape; 6. Pallet.

[0042] 11. Standard cigarette boxes - including various specifications of recycled / normal, work-in-progress / finished hard pack / soft pack / soft pack hardening / short standard cigarette boxes;

[0043] 12. Medium cigarette boxes - including boxes of recycled / normal, work-in-progress / finished medium cigarettes of various specifications;

[0044] 13. Slim cigarette boxes - including slim cigarette boxes of various specifications, including recycled / normal, work-in-progress / finished products.

[0045] 21. 10-piece stack of standard cigarette boxes - including film wrapped or unwrapped;

[0046] 22. 20-piece stack of standard cigarette boxes - including wrapped or unwrapped;

[0047] 23. 24-piece stack of standard cigarette boxes - including wrapped / unwrapped;

[0048] 24. 28-piece stack of standard cigarette boxes - including film wrapped / unwrapped;

[0049] 25. 30-piece stack of standard cigarette boxes - including wrapped / unwrapped;

[0050] 26. Empty pallet stacking type;

[0051] 27. 28-piece stack of medium cigarette boxes - including wrapped / unwrapped;

[0052] 28. Stacks of 28 slim cigarette boxes - with or without film wrapping.

[0053] 31. Example drawings of the first-tier chimney stack from the main view and the top view;

[0054] 32. Example drawings of the first and second-floor chimney stacks from the main and top views;

[0055] 33. Example drawings of the first, second and third-story chimney stacks from the main and top views;

[0056] 34. Example pictures of the first, second, third and fourth floor chimney stacks from the main view and top view. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and beneficial effects of the present invention more clear, preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings to facilitate understanding by technicians.

[0058] A method for identifying features of finished cigarette boxes based on machine vision uses a smart camera to identify whether there are physical objects in the warehouse, whether they are wrapped, the specifications, whether the boxes are recycled, the number of boxes, work-in-progress / finished products, and other content requirements, and compares them with inventory information to complete the inventory work. The characteristics of the inventory boxes required for inventory counting include: the presence of cigarette stacks in the warehouse (refer to Figure 2 ), empty storage space inventory information, pallet group inventory information, reflected light spots after cigarette stacks are wrapped, characters, patterns, colors, and barcodes on cigarette boxes (refer to Figure 1 ,1), Recycling white labels on cigarette boxes (refer to Figure 1 ,2) Marks on fixed positions (refer to Figure 1,4), main view and top view of cigarette box stack (refer to Figure 3 ), Cigarette Box No. 1 Project Barcode (refer to Figure 1 ,3) etc.

[0059] Method for identifying whether there are physical objects in the storage location: extract the colors of various cigarette boxes when there are cigarette stacks in the storage location to build a model, and then search for the color corresponding to the model in the region of interest (ROI). If it is found, it means there is a cigarette stack in the storage location, otherwise it is an empty storage location.

[0060] Method for identifying whether a cigarette stack is wrapped: When a cigarette stack is wrapped, the captured image will have a bright spot under the lighting angle. Based on this characteristic, a grayscale histogram analysis algorithm is used. The color image is converted point by point into a grayscale image in the area of interest. The grayscale histogram structure data of this area is then calculated and analyzed. The contrast within this structure data is used to determine whether the stack is wrapped. Contrast refers to the measurement of the difference in brightness between the brightest white and the darkest black in the light and dark areas of an image. A larger difference indicates greater contrast, while a smaller difference indicates less contrast.

[0061] The images of empty cargo spaces and pallet groups have a significant impact on the effect of identifying whether there are physical objects in the cargo spaces. Therefore, inventory information of empty cargo spaces and pallet groups is needed to improve the recognition accuracy.

[0062] Identification method of product specifications (refer to Figure 1 ):

[0063] (1) Pattern training: The process is as follows: select the training pattern, set the training area and origin, set the training parameters, train the pattern, and evaluate the trained features.

[0064] Pattern model training principles: select a representative pattern with consistent features; reduce unnecessary features and image noise; only train important features; consider masks to create specific patterns; larger patterns will provide higher accuracy; the more dividing points, the higher the accuracy.

[0065] (2) Runtime algorithm: Set runtime parameters, define the search area, obtain the runtime pattern, run the smart camera algorithm, and obtain the results.

[0066] Method for identifying whether a cigarette box is recycled: The recycling box logo will be affixed with a white label during the production process (refer to Figure 1 ,2) and write numbers in fixed positions (refer to Figure 1 ,4) way to mark.

[0067] (1) Using a pattern matching algorithm, we first build a white label pattern model and search for matching targets in the image. If the value is 1, it means there is a label, which means it is a recycled cigarette box; if the value is 0, it means there is no label, which means it is a new cigarette box.

[0068] (2) For the method of writing numbers in a fixed position, the OCR character recognition algorithm is used to read in a fixed ROI. The number read out is the number of times the cigarette box has been recycled.

[0069] Method for identifying the number of stack-type cigarette boxes (refer to Figure 4 ):

[0070] (1) Collect images of each stack type and each position model and perform image processing;

[0071] (2) Establish the main view model (full stack, each layer) and the top view model (each layer, each position);

[0072] (3) Real-time image processing of cargo locations;

[0073] (4) Determine whether the main view is full. If so, obtain the corresponding stack type and total quantity based on the main view full stack model. Otherwise, proceed to the next step.

[0074] (5) Determine whether the main view has four layers. If so, count the number of the fourth layer + the number of the first three layers based on the top view. Otherwise, proceed to the next step.

[0075] (6) Determine whether the main view has three layers. If so, count the number of the third layer + the number of the first two layers based on the top view. Otherwise, proceed to the next step.

[0076] (7) Determine whether the main view is the second layer. If so, count the number of the second layer + the number of the previous layer based on the top view. Otherwise, proceed to the next step.

[0077] (8) Determine whether the main view is one layer. If so, count the number of the first layer based on the top view. Otherwise, determine it as an empty storage location or pallet group.

[0078] Take the standard 28-piece stack of cigarette boxes as an example (refer to Figure 3 ):

[0079] (1) The first layer contains less than 10 cigarettes, and the placement and shape of each cigarette. The camera takes an image from a 45-degree angle directly above. Then, the image is processed (region segmentation, filtering, equalization, etc.) to build a model of the cigarette at each position to determine whether there is a cigarette at each position. Then, based on the recognition results, the total number of cigarettes in the first layer is counted. Similarly, a model of the first layer should be built on the front to determine whether it is the first layer. (Refer to Figure 3 ,31).

[0080] (2) The placement and shape of each cigarette in the second layer, which contains less than 10 pieces, is processed (region segmentation, filtering, equalization, etc.) to build a top-down model of each cigarette piece and a front-facing model of the second layer. The program first determines whether it is the second layer. If it is the second layer, it means that there are 10 pieces in the second layer. Then the number of the second layer is counted, and the sum of the two layers is the total number of the current shelf. (Refer to Figure 3 ,32).

[0081] (3) The placement and shape of each cigarette in the third layer, which contains less than 4 pieces, is processed (region segmentation, filtering, equalization, etc.) to build a top-down model of each cigarette piece and a front-view model of the third layer. The program first determines whether it is the third layer. If it is the third layer, it means that there are 10 pieces on the first and second layers, for a total of 20 pieces. Then the number of the third layer is counted, and the sum of the three layers is the total number of the current shelf. (Refer to Figure 3 ,33).

[0082] (3) The placement and shape of each cigarette in the fourth layer, which contains less than 4 pieces, is processed (regional segmentation, filtering, equalization, etc.) to establish a top-down model of each cigarette piece and a front-facing model of the third layer. The program first determines whether it is the fourth layer. If it is the fourth layer, it means that there are 10 pieces on the first and second layers, and 4 pieces on the third layer, for a total of 24 pieces. Then the number of the fourth layer is counted, and the sum of the four layers is the total number of the current shelf. (Refer to Figure 3 ,34).

[0083] Identification method of work in process / finished product (refer to Figure 1 WIP does not have a unique engineering code, while finished products do. The WIP / finished product recognition algorithm first creates a WIP pattern model, locates the target within the image, and then reads the barcode at that location. Multiple results are ORed together. If any result is 1, the product is a finished product; if any result is 0, the product is WIP.

[0084] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present invention.

Claims

1. A machine vision-based method for identifying features in finished cigarette boxes uses an intelligent camera to identify the presence of physical objects in the warehouse, whether they are wrapped, the specifications, whether the boxes are recycled, the number of boxes, and the content requirements of work-in-progress / finished products. This method then compares the information with inventory information to complete inventory inventory. It is characterized by: The method includes a method for identifying whether there are physical objects in the cargo space; a method for identifying whether the cigarette stack is wrapped; a method for identifying the specifications of the cigarettes; and a method for identifying whether the cigarette boxes are recycled. Method for identifying the number of stack-type cigarette boxes; Identification method of work in progress / finished product; The inventory cigarette box features required for the method include: information on the presence of cigarette stacks in the storage location, inventory information on empty storage locations, inventory information on pallet groups, reflected light spots after the cigarette stacks are wrapped, characters, patterns, colors, and barcodes on the cigarette boxes, white labels on recycled cigarette boxes, markings on fixed locations, front and top views of the cigarette box stacks, and the cigarette box No. 1 project barcode; The method for identifying whether there are physical objects in a cargo location is as follows: extracting the colors of various cigarette boxes when there are cigarette stacks in the cargo location to establish a model, and then searching for the color corresponding to the model in the region of interest (ROI). If a color is found, it means there is a cigarette stack in the cargo location, otherwise it is an empty cargo location. The images of empty cargo locations and pallet groups have a greater impact on the effect of identifying whether there are physical objects in the cargo location. Therefore, inventory information of empty cargo locations and pallet groups is required to improve the recognition accuracy. The method for identifying whether a cigarette stack is wrapped with film is as follows: after the cigarette stack is wrapped with film, the collected image will have a very bright light spot under the lighting angle. Based on this feature, a grayscale histogram analysis algorithm is used to convert the color image into a grayscale image point by point in the area of interest, and then the grayscale histogram structure data of this area is calculated and analyzed. The contrast in the structure data is used to determine whether the cigarette stack is wrapped with film; Method for identifying the product specifications: Step (1) Pattern training, the process is: select the training pattern, set the training area and origin, set the training parameters, train the pattern, evaluate the trained features, the pattern model training principles are: select a representative pattern with consistent features; reduce unnecessary features and image noise; only train important features; Step (2) Runtime algorithm: set runtime parameters, define the search area, obtain the runtime pattern, run the smart camera algorithm, and obtain the results.

2. The method for identifying features of a finished cigarette box based on machine vision according to claim 1, characterized in that: The method for identifying the number of stack-type cigarette boxes is as follows: Step (1) collecting images of each stack type and each position model and performing image processing; Step (2) Establishing the main view model and the top view model; Step (3) real-time image processing of cargo locations; Step (4) determines whether the main view is full of stacks. If so, obtain the corresponding stack type and total quantity according to the main view full stack model, otherwise proceed to the next step; Step (5) determines whether the main view has four layers. If so, count the number of the fourth layer + the number of the first three layers based on the top view. Otherwise, proceed to the next step. Step (6) determines whether the main view has three layers. If so, count the number of the third layer + the number of the first two layers based on the top view. Otherwise, proceed to the next step. Step (7) determines whether the main view is the second layer. If so, count the number of the second layer + the number of the previous layer based on the top view, otherwise proceed to the next step; Step (8) determines whether the main view is one layer. If so, the number of the first layer is counted based on the top view. Otherwise, it is determined to be an empty storage location or pallet group.

3. The method for identifying features of a finished cigarette box based on machine vision according to claim 1, wherein: The WIP / finished product identification method is as follows: based on the fact that WIP does not have a No. 1 engineering code, while finished products have a No. 1 engineering code, the WIP / finished product identification algorithm first establishes a pattern model of the WIP, finds the target location within the image, then reads the barcode at the location, performs an OR operation on multiple results, and if the result is 1, it is a finished product; If it is 0, it is work in progress.

Images