Tobacco leaf grasping method based on centroid detection

Abstract

The present invention relates to the technical field of tobacco leaf processing, and more particularly to a tobacco leaf grasping method based on centroid detection, comprising: step S1: capturing a tobacco leaf image, extracting a leaf vein portion from the tobacco leaf image to obtain a leaf vein image; step S2: calculating the centroid of the leaf vein portion in each grasping area based on the leaf vein image and generating centroid coordinates corresponding to the grasping area; and step S3: grasping the tobacco leaf in each grasping area using the centroid coordinates. The present invention has the beneficial effect of effectively determining the position of the tobacco leaf in a silo by capturing a tobacco leaf image, extracting the leaf centroid based on the leaf vein portion in the tobacco leaf image, and then grasping the tobacco leaf based on the centroid coordinates.

Description

Technical Field

[0001] The present invention relates to the technical field of tobacco leaf processing, and in particular to a tobacco leaf grabbing method based on centroid detection. Background Art

[0002] The tobacco production process often involves multiple steps, including weighing, grading, sorting, and baking. Between these steps, tobacco leaves often need to be grasped and moved to coordinate between different processes.

[0003] In existing technology, tobacco leaf grabbing and loading operations are typically performed using a robot. This type of technical solution typically includes a loading mechanism, such as an articulated gripper or suction cup, which reciprocates between different devices using coordinate parameters pre-programmed in a PLC device to grab the tobacco leaves.

[0004] However, during actual implementation, the inventors discovered that, because the position of the tobacco leaves to be grasped in the silo or hopper is not fixed, grasping them using only the fixed coordinate parameters pre-programmed in the PLC device often fails to accurately grasp the leaves. Furthermore, tobacco leaves tend to fold when stacked in the silo, which results in existing grasping methods being unable to effectively grasp the leaves, or causing the leaves to slip during grasping due to inaccurate grasping positions. Summary of the Invention

[0005] In view of the above problems existing in the prior art, a tobacco leaf grabbing method based on centroid detection is provided.

[0006] The specific technical solutions are as follows:

[0007] A tobacco leaf grabbing method based on centroid detection uses a grabbing device to grab tobacco leaves, wherein the grabbing device is provided with at least one grabbing area. The tobacco leaf grabbing method includes:

[0008] Step S1: collecting a tobacco leaf image, and extracting a vein portion from the tobacco leaf image to obtain a vein image;

[0009] Step S2: calculating the centroid of the leaf vein portion in each of the grasping areas according to the leaf vein image and generating the centroid coordinates corresponding to the grasping area;

[0010] Step S3: In each of the grabbing areas, the tobacco leaves are grabbed using the centroid coordinates.

[0011] Preferably, the step S1 includes:

[0012] Step S11: collecting the tobacco leaf image;

[0013] Step S12: using a neural network model to segment the vein position of the vein portion from the tobacco leaf image to generate a segmented image;

[0014] Step S13: Processing the segmented image to obtain the leaf vein image.

[0015] Preferably, the step S11 includes:

[0016] Step S111: using an image sensor to capture a tobacco leaf photo from above the tobacco leaf;

[0017] Step S112: performing distortion correction on the tobacco leaf photo to generate the tobacco leaf image.

[0018] Preferably, the step S13 includes:

[0019] Step S131: removing noise from the segmented image to generate a noise-reduced image;

[0020] Step S132: performing morphological processing on the leaf vein portion in the denoised image according to the leaf vein position to generate a pre-processed image;

[0021] Step S133: binarizing the pre-processed image to generate the leaf vein image.

[0022] Preferably, in step S133, the binarization processing method includes:

[0023]

[0024] Among them, src_img is the leaf vein image, bin_img is the preprocessed image, img_val is the feature output value of a single pixel in the preprocessed image, low_val is the preset feature value lower limit, and up_val is the preset feature value upper limit.

[0025] Preferably, step S2 includes:

[0026] Step S21: segmenting the leaf vein image according to the captured area to generate a partition image;

[0027] Step S22: In the partitioned image, respectively calculating the centroid of the leaf vein in each of the captured areas to generate a plurality of centroid coordinates.

[0028] Preferably, the step S22 includes:

[0029] Step S221: for each of the captured areas, generating a minimum circumscribed rectangle for the leaf vein portion;

[0030] Step S222: Generate the centroid of each leaf vein in the grasping area according to the minimum circumscribed rectangle to obtain the centroid coordinates of the grasping area.

[0031] Preferably, the gripping device includes a suction cup array, each of the gripping areas of the suction cup array has a suction cup, and the suction cup moves along the longitudinal and / or transverse axis in the gripping area to suck the tobacco leaves according to the center of mass coordinates.

[0032] Preferably, step S3 includes:

[0033] Step S31: for each of the suction cups, respectively obtain the current coordinates of the suction cup;

[0034] Step S32: Calculating the movement vector of each suction cup according to the current coordinates and the coordinates of the centroid of the gripping area corresponding to the suction cup;

[0035] Step S33: using the movement vector to control the suction cups respectively to grab the tobacco leaves.

[0036] Preferably, in step S33, the movement vector is linearly scaled to obtain actual movement parameters of the suction cup, and the position of the suction cup is adjusted according to the actual movement parameters to grab the tobacco leaves.

[0037] The above technical solution has the following advantages or beneficial effects: by collecting tobacco leaf images and extracting the center of mass of the tobacco leaves based on the vein part in the tobacco leaf images, and then grabbing them according to the center of mass coordinates, it is possible to effectively judge the position of the tobacco leaves in the silo and achieve a relatively stable grabbing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The embodiments of the present invention will be described more fully with reference to the accompanying drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0039] Figure 1 is an overall schematic diagram of an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram of sub-steps of step S1 in an embodiment of the present invention;

[0041] Figure 3 Schematic diagram of tobacco leaf image in an embodiment of the present invention;

[0042] Figure 4 Schematic diagram of a leaf vein image in an embodiment of the present invention;

[0043] Figure 5This is a schematic diagram of sub-steps of step S11 in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of sub-steps of step S13 in an embodiment of the present invention;

[0045] Figure 7 This is a schematic diagram of the sub-steps of step S2 in an embodiment of the present invention;

[0046] Figure 8 Schematic diagram of image segmentation in an embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of sub-steps of step S22 in an embodiment of the present invention;

[0048] Figure 10 Schematic diagram of the minimum circumscribed rectangle in an embodiment of the present invention;

[0049] Figure 11 Schematic diagram of sub-steps of step S3 in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0051] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0053] The present invention comprises:

[0054] The tobacco leaf grabbing method based on centroid detection adopts a grabbing device to grab tobacco leaves, and the grabbing device is provided with at least one grabbing area, such as Figure 1 As shown, the tobacco leaf grabbing method includes:

[0055] Step S1: collecting a tobacco leaf image, and extracting a leaf vein portion from the tobacco leaf image to obtain a leaf vein image;

[0056] Step S2: Calculate the centroid of the leaf vein portion in each grasping area according to the leaf vein image and generate the centroid coordinates corresponding to the grasping area;

[0057] Step S3: In each grabbing area, grab the tobacco leaves using the centroid coordinates.

[0058] Specifically, to address the problem of existing grasping solutions failing to accurately grasp tobacco leaves, this embodiment captures a tobacco leaf image and extracts the leaf veins from the image to determine its center of mass. The leaf can then be grasped based on the center of mass coordinates. Because the center of mass of the tobacco leaf is acquired through image recognition during grasping, the grasping process ensures a relatively stable grasp, avoiding issues such as unstable grasping and leaf drop caused by the additional deflection torque generated by the leaf's own weight.

[0059] During implementation, the above method, as a software embodiment, is implemented in a corresponding gripping device. For example, in tobacco production, tobacco leaves need to be removed from a silo and transferred to a curing workshop for curing. In this case, the gripping device can be controlled to grip the tobacco leaves according to the above tobacco gripping method, thereby achieving more stable gripping. In one embodiment, the gripping device is a suction cup-shaped gripping device, which controls the suction cup to move above the tobacco leaf's center of mass to grasp the leaf's center of mass, thereby achieving a more stable gripping effect. In other embodiments, the gripping device can be replaced with other similar devices as needed, such as a claw-shaped gripping device that aligns the center of the claw with the tobacco leaf's center of mass to prevent the leaf from slipping. Tobacco leaf images are captured using an image sensor, which is positioned directly above the gripping position and pointed downward at the tobacco leaf to achieve better image quality. In some embodiments, the image sensor can be used in conjunction with other devices, such as an auxiliary light source, to provide fill light for capturing the tobacco leaf, thereby capturing a clearer image of the tobacco leaf.

[0060] In a preferred embodiment, Figure 2 As shown, step S1 includes:

[0061] Step S11: collecting tobacco leaf images;

[0062] Step S12: using a neural network model to segment the vein positions of the leaf vein portion from the tobacco leaf image to generate a segmented image;

[0063] Step S13: Process the segmented image to obtain a leaf vein image.

[0064] Specifically, in the prior art, due to the random placement of tobacco leaves and the possibility of folding, the specific location of the tobacco leaves to be grasped cannot be accurately identified, thereby reducing the grasping accuracy. In this embodiment, the position of the leaf vein portion is extracted from the tobacco leaf image to generate a leaf vein image, thereby achieving the positioning of the tobacco leaf by the leaf vein. Since the leaf vein is always located in the central area of the tobacco leaf, the leaf vein positioning method can achieve a better positioning effect. At the same time, since only the leaf vein portion is identified during the extraction process, it reduces the number of features to be extracted compared to the identification process of the entire tobacco leaf, thereby improving the recognition efficiency.

[0065] During the implementation process, if Figure 3 As shown in FIG, for a tobacco leaf A to be grasped, the vein portion A1 of the tobacco leaf A to be grasped can be obtained by collecting the tobacco leaf image. At this time, by inputting the tobacco leaf image into the neural network model, the neural network model pre-trained for the vein features can recognize and segment the vein portion A1. For example, in one embodiment, a semantic recognition model is used for recognition, and the vein portion A1 is used as the foreground. The leaf portion and the background image of the tobacco leaf are both segmented as the background portion, thereby obtaining the position of the vein portion to generate a segmented image. Subsequently, the segmented image is processed to retain only the vein portion A1, thereby obtaining the following image: Figure 4 To achieve better processing efficiency, the image can be binarized to retain only the position information of the leaf vein part A1 in the image.

[0066] In a preferred embodiment, Figure 5 As shown, step S11 includes:

[0067] Step S111: using an image sensor to capture a tobacco leaf photo from above the tobacco leaf;

[0068] Step S112: performing distortion correction on the tobacco leaf photo to generate a tobacco leaf image.

[0069] Specifically, in the prior art, due to the random placement of tobacco leaves and the possibility of folding, the specific position of the tobacco leaves to be grasped cannot be accurately identified, which in turn causes a decrease in grasping accuracy. In this embodiment, an image sensor is set directly above the tobacco leaves to collect tobacco leaf photos, and then the tobacco leaf photos are corrected for lens distortion based on the lens parameters of the image sensor. At the same time, the tobacco leaf image is reprojected onto the grasping plane of the grasping device, so that the position information in the tobacco leaf image can correspond to the coordinate parameters of the grasping device, so as to achieve a more accurate grasping effect.

[0070] In a preferred embodiment, Figure 6 As shown, step S13 includes:

[0071] Step S131: removing noise from the segmented image to generate a denoised image;

[0072] Step S132: performing morphological processing on the leaf vein portion in the denoised image according to the leaf vein position to generate a pre-processed image;

[0073] Step S133: binarize the pre-processed image to generate a leaf vein image.

[0074] Specifically, in view of the problem in the prior art that the specific position of the tobacco leaves to be grasped cannot be accurately identified due to the random placement of the tobacco leaves and the possibility of folding, which in turn causes a decrease in grasping accuracy, in this embodiment, after identifying the position of the leaf vein part, the noise of the segmented image is removed to avoid the influence of the image noise on subsequent processing. Subsequently, the boundary between the leaf vein part and the "background" part is sharpened through open morphological processing to obtain a clearer edge contour, and then the image is binarized to obtain the following image. Figure 4 A clearer image of leaf veins is shown.

[0075] In a preferred embodiment, in step S133, the binarization processing method includes:

[0076]

[0077] Among them, src_img is the leaf vein image, bin_img is the preprocessed image, img_val is the feature output value of a single pixel in the preprocessed image, low_val is the preset lower limit of the feature value, and up_val is the preset upper limit of the feature value.

[0078] Specifically, in order to achieve a better effect of extracting the leaf vein part, in this embodiment, for the output results of the neural network model, by setting the upper limit and the lower limit of the eigenvalue respectively, a better classification of the image features is achieved, so that the image features of the leaf vein part are accurately extracted from the image features, and binarization processing is performed according to the classification results, thereby simplifying the image information and retaining only the position information of the leaf vein part in the image, thereby achieving better processing efficiency.

[0079] In a preferred embodiment, a suction cup array is used to grab tobacco leaves. The suction cup array is provided with a plurality of suction cups, each of which has a grabbing area, and the suction cups can move within the grabbing area.

[0080] Then Figure 7 As shown, step S2 includes:

[0081] Step S21: segmenting the leaf vein image according to the captured area to generate a partition image;

[0082] Step S22: Calculate the centroid of the leaf vein in each captured area in the partitioned image to generate a plurality of centroid coordinates.

[0083] Specifically, in order to address the problem that the grasping device in the prior art relies on fixed coordinate parameters for grasping during the grasping process, resulting in unsatisfactory grasping effect and inability to accurately grasp the tobacco leaves, in this embodiment, the grasping device is set as a suction cup array, and the tobacco leaves are divided according to the grasping areas on the suction cup array. Each suction cup corresponds to a grasping area and moves in its respective grasping area to grasp the centroid of the leaf veins of that part, thereby achieving a more stable grasping effect.

[0084] In one embodiment, if Figure 8 As shown, the gripping device is configured as a four-partitioned suction cup array, wherein each suction cup can move within a gripping area. In this embodiment, the suction cup is a long strip suction cup, the length of its long axis being equal to the length of the gripping area A2, and the suction cup only moves back and forth along the longitudinal axis in the gripping area A2. In other embodiments, there are other types of suction cups, such as a long strip suction cup that moves laterally along the gripping area A2, a suction cup that moves horizontally in the transverse or longitudinal direction at the same time, and a suction cup matrix that can be individually controlled for suction. In order to achieve a more stable gripping effect, at this time, on the basis of reprojecting the tobacco leaf image onto the gripping plane of the gripping device, the gripping area A2 is directly mapped to the tobacco leaf image, thereby obtaining the following: Figure 8 For example, in one embodiment, the coordinates of the capture area in the partition image are set as shown below:

[0085]

[0086] Among them, rio_pos represents the coordinate parameters of the grab area A2, index_xy represents the pre-set coordinate range of the grab area, tlx represents the horizontal axis coordinate range of the grab area A2, tly represents the vertical axis coordinate range of the grab area A2, ind_x is the origin of the horizontal axis, ind_y is the origin of the vertical axis, height represents the vertical axis length of the grab area A2, and width represents the horizontal axis length of the grab area A2.

[0087] The grabbing area A2 is a rectangular grabbing area of 450*350, and its actual horizontal axis length and vertical axis length can be adjusted according to the mapping result of the tobacco leaf image by the grabbing device in different embodiments.

[0088] In a preferred embodiment, Figure 9 As shown, step S22 includes:

[0089] Step S221: For each captured area, generate a minimum bounding rectangle for the leaf vein portion;

[0090] Step S222: Generate the centroid of the leaf vein in each grasping area according to the minimum circumscribed rectangle to obtain the centroid coordinates of the grasping area.

[0091] Specifically, in order to achieve a better extraction effect of the centroid of the leaf vein and a faster processing speed, in this embodiment, a minimum circumscribed rectangle is used in each capture area to fit the capture area to form a Figure 10 The fitting image shown is then used to obtain the centroid of the leaf vein based on the length and width of the minimum circumscribed rectangle to achieve better processing efficiency.

[0092] In one embodiment, the gripping suction cup is configured as a long strip of suction cup that moves only along the longitudinal axis. To achieve a simpler center of mass calculation, only the longitudinal axis coordinate needs to be extracted, as shown in the following formula:

[0093]

[0094] Where rect_c_point is the extracted center of mass coordinate, cx is the horizontal coordinate of the center of mass, cy is the vertical coordinate of the center of mass, rect_tlx is the horizontal value of the origin of the minimum bounding rectangle, rect_tly is the vertical value of the origin of the minimum bounding rectangle, height represents the vertical length of the gripping area A2, and width represents the horizontal length of the gripping area A2. In this embodiment, since only the vertical movement of the suction cup is required, only the vertical coordinate portion, i.e., cy in the above formula, needs to be extracted.

[0095] In a preferred embodiment, Figure 10 As shown, step S3 includes:

[0096] Step S31: for each suction cup, obtain the current coordinates of the suction cup respectively;

[0097] Step S32: Calculating the movement vector of each suction cup according to the current coordinates and the coordinates of the centroid of the gripping area corresponding to the suction cup;

[0098] Step S33: using the movement vector to control the suction cups respectively to grab the tobacco leaves.

[0099] Specifically, in order to achieve a better grasping effect, in this embodiment, the current coordinates of each suction cup are obtained respectively, and the corresponding movement vector is generated by combining the current coordinates of the suction cup and the center of mass coordinates in the grasping area, and the movement vector is used for grasping, thereby achieving a better grasping effect.

[0100] In a preferred embodiment, in step S33, the movement vector is linearly scaled to obtain actual movement parameters of the suction cup, and the position of the suction cup is adjusted according to the actual movement parameters to grab the tobacco leaves.

[0101] Specifically, to achieve a better grasping effect, in this embodiment, the grasping coordinates obtained based on the image are linearly scaled so that the generated motion vector conforms to the actual motion parameters of the grasping array, specifically as follows: delat_y = K*(act_y-ori_y);

[0102] Where delat_y is the actual movement parameter, K is the scaling factor determined based on the tobacco leaf image, act_y is the zero coordinate position of the suction cup, and ori_y is the coordinate of the center of mass in the area. In one embodiment, a scaling factor of 0.1 is used for an image with a resolution of 1000*600.

[0103] The beneficial effects of the present invention are as follows: In response to the existing technology's use of relatively fixed parameters for grasping, which can lead to tobacco leaves being misplaced and potentially folding in the silo, resulting in failure to grasp the leaves or causing them to slip during grasping, the present invention captures an image of the tobacco leaf to be grasped in real time and identifies the leaf veins from the image, enabling the grasping device to determine the leaf's center of mass based on the veins and grasp it accordingly, achieving a more stable grasping effect. Furthermore, by adjusting the grasping device to generate multiple grasping areas, each of which is equipped with a movable suction cup, the number of suction cups in the suction cup array is reduced, lowering equipment costs.

[0104] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A tobacco leaf grabbing method based on centroid detection, characterized in that: A gripping device is used to grip tobacco leaves, wherein the gripping device is provided with at least one gripping area, and the gripping device includes a suction cup array, wherein each gripping area of the suction cup array has a suction cup, and the suction cup moves along the longitudinal axis and / or the transverse axis in the gripping area; The tobacco leaf grabbing method comprises: Step S1: collecting a tobacco leaf image, and extracting a vein portion from the tobacco leaf image to obtain a vein image; Step S2: calculating the centroid of the leaf vein portion in each of the grasping areas according to the leaf vein image and generating the centroid coordinates corresponding to the grasping area; Step S3: In each of the grabbing areas, each of the suction cups uses the centroid coordinates to grab the vein portion of the tobacco leaf; The step S2 comprises: Step S21: segmenting the leaf vein image according to the captured area to generate a partition image; Step S22: calculating the centroid of each leaf vein in the captured area in the partitioned image to generate a plurality of centroid coordinates; The step S22 includes: Step S221: for each of the captured areas, generating a minimum circumscribed rectangle for the leaf vein portion; Step S222: generating the centroid of each leaf vein in the grasping area according to the minimum circumscribed rectangle to obtain the centroid coordinates of the grasping area; The centroid coordinates are extracted according to the following formula: Among them, rect_c_point is the extracted center of mass coordinate, cx is the horizontal axis coordinate of the center of mass, cy is the vertical axis coordinate of the center of mass, rect_tlx is the horizontal axis value of the origin of the minimum bounding rectangle, rect_tly is the vertical axis value of the origin of the minimum bounding rectangle, height represents the vertical axis length of the grasping area, and width represents the horizontal axis length of the grasping area.

2. The tobacco leaf grabbing method according to claim 1, characterized in that: The step S1 comprises: Step S11: collecting the tobacco leaf image; Step S12: using a neural network model to segment the vein position of the vein portion from the tobacco leaf image to generate a segmented image; Step S13: Processing the segmented image to obtain the leaf vein image.

3. The tobacco leaf grabbing method according to claim 2, characterized in that: The step S11 includes: Step S111: using an image sensor to capture a tobacco leaf photo from above the tobacco leaf; Step S112: performing distortion correction on the tobacco leaf photo to generate the tobacco leaf image.

4. The tobacco leaf grabbing method according to claim 2, characterized in that: The step S13 includes: Step S131: removing noise from the segmented image to generate a noise-reduced image; Step S132: performing morphological processing on the leaf vein portion in the denoised image according to the leaf vein position to generate a pre-processed image; Step S133: binarizing the pre-processed image to generate the leaf vein image.

5. The tobacco leaf grabbing method according to claim 4, characterized in that: In step S133, the binarization processing method includes: Among them, src_img is the leaf vein image, bin_img is the preprocessed image, img_val is the feature output value of a single pixel in the preprocessed image, low_val is the preset feature value lower limit, and up_val is the preset feature value upper limit.

6. The tobacco leaf grabbing method according to claim 1, characterized in that: The step S3 comprises: Step S31: for each of the suction cups, respectively obtain the current coordinates of the suction cup; Step S32: Calculating the movement vector of each suction cup according to the current coordinates and the coordinates of the centroid of the gripping area corresponding to the suction cup; Step S33: using the movement vector to control the suction cups respectively to grab the tobacco leaves.

7. The tobacco leaf grabbing method according to claim 6, characterized in that: In step S33, the movement vector is linearly scaled to obtain actual movement parameters of the suction cup, and the position of the suction cup is adjusted according to the actual movement parameters to grab the tobacco leaves.

Images