A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage

Abstract

The present invention belongs to the technical field of auxiliary material storage, and discloses an auxiliary material quantity identification method for the storage of auxiliary materials for cigarettes. The method comprises the following steps: pre-processing to remove background and noise; point cloud segmentation; material shape fitting; and inference statistics based on height information. The present invention optimizes the auxiliary material storage process, improves inventory efficiency, reduces unnecessary working time, reduces labor costs, improves production supply efficiency, and provides strong support for the intelligent management of the rolling and packaging workshop.

Description

Technical Field

[0001] The present invention relates to the technical field of auxiliary material storage, and in particular to an auxiliary material quantity identification method for auxiliary material storage of cigarettes. Background Art

[0002] The distribution of auxiliary materials in the cigarette industry is divided into two forms: manual and automatic. However, the automatic auxiliary material return process has not been fully intelligent and automated, and still uses manual counting, which has the characteristics of low counting efficiency, high labor intensity, low accuracy, energy waste, and easy errors.

[0003] According to the preliminary investigation, there are very few companies in the industry that conduct intelligent inventory of auxiliary materials. The existing system can only calculate the number of coils or tipping papers, lacks real-time transmission and display functions, and cannot interact with the WCS (Warehouse Control System) / WMS (Warehouse Management System) system for data exchange. The existing system requires manual operation of the controller and lacks automation. There is a certain degree of human interference and error in the inventory process. Although it can improve the efficiency of warehousing, it still has problems such as long inventory time and low efficiency in large-scale production environments. Summary of the invention

[0004] The purpose of the present invention is to provide a method for identifying the quantity of auxiliary materials for cigarette auxiliary materials when they are put into storage, so as to solve the problem.

[0005] To achieve this object, the present invention adopts the following technical solutions:

[0006] A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage, the method comprising the following steps:

[0007] Preprocessing to remove background and noise;

[0008] Point cloud segmentation;

[0009] Material shape fitting;

[0010] Based on highly informative inference statistics.

[0011] Preferably, the preprocessing to remove background and noise includes: acquiring point cloud data through a 3D camera, setting a spatial range where the target material is located, and treating point cloud data outside the spatial range as invalid point cloud and removing it.

[0012] Preferably, the point cloud segmentation includes: performing semantic segmentation on the point cloud data according to the category results identified in the auxiliary material RGB image collected by the camera, and segmenting different materials contained in the point cloud data to form independent point clouds.

[0013] Preferably, the step of forming independent point clouds comprises:

[0014] Image depth (rect,label)|x,y|=Image rgb (rect,label)|x,y|

[0015] Among them, rect is the information of the target area box, and label is the category corresponding to the target area box.

[0016] Preferably, the material shape fitting comprises the following steps: fitting the plane in the independent point cloud using a RANSAC fitting algorithm.

[0017] Preferably, the RANSAC first randomly selects initial data points that meet known set conditions, and then uses a consistent data point set to expand the initial data points.

[0018] Preferably, the RANSAC first randomly selects initial data points that meet known set conditions, and then uses a consistent data point set to expand the initial data points, specifically including:

[0019] Select three points at random in the target point cloud and calculate their plane equations:

[0020] Ax+By+Cz=0

[0021] Calculate all points (x i ,y i ,z i ) to the plane equation:

[0022] d i =|Ax i +By i +Cz i |

[0023] Select the threshold d th , if d i ≤d th , then record the point as an internal sample point, otherwise record the point as an external sample point, and record the total number of the current left and right internal points;

[0024] Repeat the above steps and select the model parameters corresponding to the plane with the largest number of in-house sample points as the final estimated target point cloud plane parameters. The in-library set under this parameter is the target material plane point cloud.

[0025] Preferably, the inference statistics based on height information includes: taking the point cloud plane after RANSAC fitting as the accurate height of the actual material, and calculating the number of auxiliary materials in each material pile through prior knowledge of the height of each auxiliary material.

[0026] Preferably, in actual calculation, the material inference counting is realized by pre-agreing on a relationship lookup table between the height range of each auxiliary material and the number of auxiliary materials in the auxiliary material pile.

[0027] Preferably, the method for identifying the category in the auxiliary material RGB image captured by the camera is: using a deep neural network as an auxiliary material type recognition model, using the auxiliary material RGB three-dimensional image captured by the camera as input and performing convolution calculation;

[0028] The convolutional layers of each level of the deep neural network are cascaded, and the obtained image data is used to obtain a prediction box and an auxiliary material category of the prediction box through matrix calculation.

[0029] Beneficial effects of the invention: The invention realizes the intelligent calculation and recording of the auxiliary material quantity and the real-time data transmission and display functions. Compared with the traditional system, the invention optimizes the auxiliary material storage process, improves the inventory efficiency, reduces unnecessary working time, reduces labor costs, improves the production supply efficiency, and provides strong support for the intelligent management of the packaging workshop. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the contents of the embodiments of the present invention and these drawings without paying any creative work.

[0031] Figure 1 The present invention is a flow chart of a method for identifying the quantity of auxiliary materials for warehousing cigarette auxiliary materials. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0033] In the description of the present invention, unless otherwise clearly specified and limited, the terms "connected", "connected", and "fixed" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0034] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply indicates that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly below and obliquely below the second feature, or simply indicates that the first feature is lower in level than the second feature.

[0035] In the description of this embodiment, the terms "upper", "lower", "left", "right" and other directions or positional relationships are based on the directions or positional relationships shown in the drawings, and are only for the convenience of description and simplification of operation, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operate in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used to distinguish in the description and have no special meaning.

[0036] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0037] like Figure 1 As shown, the present invention provides a method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage, the method comprising the following steps:

[0038] Preprocessing to remove background and noise; the preprocessing to remove background and noise includes: the point cloud data obtained by the 3D camera contains background information such as the pallet, and there is a lot of noise, which is easy to interfere with the subsequent point cloud registration and auxiliary material segmentation and recognition work, and the background and noise need to be removed. In the background removal stage, the background is mainly removed based on the depth information of the area of ​​interest. Set the spatial range where the target material is located, and the point cloud outside the range is regarded as invalid point cloud and removed. This method can quickly remove the background area and retain the point cloud of the area of ​​interest.

[0039] Point cloud segmentation: semantically segment the point cloud data according to the category results identified in the auxiliary material RGB image collected by the camera, and segment the different materials contained in the point cloud data, such as frame paper, inner card paper, etc., one by one to form their own independent point clouds.

[0040] The type identification method comprises the following steps:

[0041] The deep neural network is used as the auxiliary material type recognition model, and the auxiliary material RGB three-dimensional image taken by the camera is used as input and convolution calculation is performed; the convolution calculation uses the maximum pooling layer as the standard for pixel value extraction. The Task-Aligned Assigner alignment assigner is used for the deep neural network to perform a dynamic allocation strategy, and the weight is dynamically adjusted according to the progress of training and the characteristics of the sample.

[0042] The dynamic adjustment weight is adjusted according to the training loss or index,

[0043] t=s α +u β ;

[0044] Among them, α and β are weight hyperparameters, s is the predicted score corresponding to the labeled category, and u is the IoU between the predicted box and the actual box. The sum of the two can measure the degree of alignment; t controls the optimization of both the classification score and IoU to achieve the degree of alignment. When the category score is higher and the IoU is higher, the value of t is closer to 1.

[0045] The alignment steps include:

[0046] Based on the classification score and the IoU between the predicted box and the actual box, a loss function of the IoU of the associated classification and regression is obtained by weighting. The IoU loss function is as follows:

[0047]

[0048] Among them, IoU represents the intersection-over-union ratio of the actual box and the predicted box; A and B represent the actual box and the predicted box respectively:

[0049]

[0050] b, b gt Represent the center points of the predicted box and the actual box respectively, ρ represents the Euclidean distance between the center points of the predicted box and the actual box, c represents the diagonal distance of the minimum closure area that can contain both the predicted box and the actual box, and v is used to measure the similarity of the aspect ratio, which is defined as:

[0051]

[0052] w, w gt Represent the width of the predicted box and the width of the actual box, h, h gt Represent the height value of the predicted box and the height value of the actual box respectively;

[0053] Calculate whether the center point of the anchor point is within the current actual box. Only the anchor point value within the actual box can be used as a positive sample.

[0054] Based on the alignment score IoU, the points in the actual box are selected as positive samples, and the rest are used as negative samples for training.

[0055] The steps of forming independent point clouds include:

[0056] Image depth (rect,label)|x,y|=Image rgb (rect,label)|x,y|;

[0057] Among them, rect is the information of the target area box, and label is the category corresponding to the target area box. By projecting the target box in RGB onto the depth map, point cloud computing is performed on the area within the target box corresponding to the depth map, so that only the point cloud within the target box is calculated, and effective segmentation of the point cloud is achieved.

[0058] The convolutional layers at each level are cascaded, and the obtained image data is calculated through matrix calculation to obtain the auxiliary material categories of the predicted box and the actual box.

[0059] Material shape fitting; the material shape fitting includes the following steps: using the RANSAC fitting algorithm to fit the plane in the independent point cloud. According to the different shapes of the auxiliary materials, it is divided into space circle fitting, plane fitting, etc. RANSAC is a random parameter estimation algorithm. RANSAC divides the data points into internal points and external points. The ultimate goal is to eliminate external points as much as possible and retain internal points to the maximum extent. It is different from other similar algorithms in that RANSAC does not use all the original point cloud data to obtain the initial solution and then remove outliers. The RANSAC first randomly selects the initial data points that meet the known set conditions, and then uses a consistent data point set to expand the initial data points. The RANSAC first randomly selects the initial data points that meet the known set conditions, and then uses a consistent data point set to expand the initial data points, specifically including:

[0060] Select three points at random in the target point cloud and calculate their plane equations:

[0061] Ax+By+Cz=0;

[0062] Calculate all points (x i ,y i ,z i ) to the plane equation:

[0063] d i =|Ax i +By i +Cz i |;

[0064] Select the threshold dth , if d i ≤d th , then record this point as an in-game sample point, otherwise record this point as an out-game sample point, and record the total number of in-game points around this point;

[0065] Repeat the above steps and select the model parameters corresponding to the plane with the largest number of in-house sample points as the final estimated target point cloud plane parameters. The in-library set under this parameter is the target material plane point cloud.

[0066] Inferring statistics based on height information. Inferring statistics based on height information includes: using the point cloud plane fitted by RANSAC as the accurate height of the actual material, and calculating the number of auxiliary materials in each material pile through the prior knowledge of the height of each auxiliary material. In actual calculation, the material inference counting is achieved by pre-agreeding the relationship between the height range of each auxiliary material and the number of auxiliary materials in the auxiliary material pile.

[0067] After the inventory is completed, the system compares the auxiliary material results of visual identification with the initial situation of the pallet provided in advance by the WCS / WMS system. The comparison process includes verifying whether the type and quantity of each material on the pallet are consistent with the expectations. Specifically, the storage area of ​​the expected material information is set in the visual system so as to compare with the actual inventory results. If any material type or quantity is found to be inconsistent with the expectations, the system will immediately trigger an alarm and display the specific information of the error on the user interface so that the worker can correct the error in time.

[0068] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, various obvious changes, readjustments and substitutions can be made without departing from the protection scope of the present invention. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage, characterized in that: The method comprises the following steps: Preprocessing to remove background and noise; Point cloud segmentation; Material shape fitting; Based on highly informative inference statistics.

2. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 1, characterized in that: The preprocessing to remove background and noise includes: acquiring point cloud data through a 3D camera, setting a spatial range where the target material is located, and treating point cloud data outside the spatial range as invalid point cloud and removing it.

3. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 2, characterized in that: The point cloud segmentation includes: performing semantic segmentation on the point cloud data according to the category results identified in the auxiliary material RGB image collected by the camera, and segmenting different materials contained in the point cloud data to form independent point clouds.

4. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 3, characterized in that: The steps of forming independent point clouds include: Image depth (rect,label)|x,y|=Image rgb (rect,label)|x,y| Among them, rect is the information of the target area box, and label is the category corresponding to the target area box.

5. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 4, characterized in that: The material shape fitting comprises the following steps: using a RANSAC fitting algorithm to fit the plane in the independent point cloud.

6. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 5, characterized in that: RANSAC first randomly selects initial data points that meet known set conditions, and then uses a consistent data point set to expand the initial data points, specifically including: Select three points at random in the target point cloud and calculate their plane equations: Ax+By+Cz=0 Calculate all points (x i ,y i ,z i ) to the plane equation: d i =|Ax i +By i +Cz i | Select the threshold d th , if d i ≤d th , then record this point as an in-game sample point, otherwise record this point as an out-game sample point, and record the total number of in-game points around this point; Repeat the above steps and select the model parameters corresponding to the plane with the largest number of in-house sample points as the final estimated target point cloud plane parameters. The in-library set under this parameter is the target material plane point cloud.

7. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 6, characterized in that: The inference statistics based on height information includes: taking the plane point cloud of the target material after RANSAC fitting as the accurate height of the actual material, and calculating the number of auxiliary materials in each material pile through the prior knowledge of the height of each auxiliary material.

8. A method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 7, characterized in that: In actual calculation, material inference counting is achieved by pre-agreed on a relationship lookup table between the height range of each auxiliary material and the number of auxiliary materials in the auxiliary material pile.

9. The method for identifying the quantity of auxiliary materials for cigarette auxiliary materials storage according to claim 3, characterized in that: The method for identifying the categories based on the auxiliary material RGB image captured by the camera is as follows: using a deep neural network as an auxiliary material type recognition model, using the auxiliary material RGB three-dimensional image captured by the camera as input and performing convolution calculation; The convolutional layers of each level of the deep neural network are cascaded, and the obtained image data is used to obtain a prediction box and an auxiliary material category of the prediction box through matrix calculation.

Images