Uppermost tobacco leaf segmentation method and system based on dense feature point matching

Abstract

The application provides a topmost tobacco leaf segmentation method and system based on dense feature points, and belongs to the technical field of tobacco leaf grading. The method comprises the following steps: S1, a first image and a second image containing a plurality of stacked tobacco leaves are acquired, the first image is an image before the topmost tobacco leaf is grabbed, and the second image is an image after the topmost tobacco leaf is grabbed; S2, dense feature point matching is performed on the tobacco leaf regions of the first image and the second image to obtain feature matching points; S3, abnormal feature matching points are removed; and S4, for the first image, each feature matching point is eroded according to a preset erosion direction, and the remaining region after erosion is the topmost tobacco leaf segmented. The application creatively uses the method of dense feature point matching to segment the topmost tobacco leaf from a stack of tobacco leaves, thereby realizing the grading processing of the topmost tobacco leaf. The segmentation method is not affected by changes in the external environment and has high robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tobacco leaf grading, and particularly relates to a topmost tobacco leaf segmentation method and system based on dense feature point matching. BACKGROUND

[0002] Tobacco leaf is an important raw material in tobacco product production, and its grade quality directly affects the quality of tobacco products. In order to provide required tobacco leaf raw materials for various tobacco products and promote the development of tobacco leaf production, the state tobacco industry formulates scientific and reasonable grading standards for tobacco leaf grading.

[0003] In the prior art, the tobacco leaf is mostly graded by manual grading, and the grading method is often limited by the experience of the grader, and the grading efficiency is low. With the development of deep learning and artificial intelligence technology, a deep learning-based method greatly solves the current tobacco leaf grading problem by relying on good features and a powerful classifier. However, in the actual production process, a stack of overlapping tobacco leaves needs to be graded, and how to segment the topmost tobacco leaf from the stack of overlapping tobacco leaves is a very difficult problem at present. SUMMARY

[0004] To solve the above technical problems, the present application provides a topmost tobacco leaf segmentation method and system based on dense feature point matching.

[0005] The technical problem solved by the present application can be realized by the following technical scheme:

[0006] The topmost tobacco leaf segmentation method based on dense feature point matching comprises the following steps:

[0007] Step S1, a first image and a second image containing a stack of multiple layers of tobacco leaves are obtained, the first image is an image before the topmost tobacco leaf is grabbed, and the second image is an image after the topmost tobacco leaf is grabbed;

[0008] Step S2, dense feature point matching is performed on the tobacco leaf regions of the first image and the second image to obtain feature matching points;

[0009] Step S3, the feature matching points that are abnormal are removed;

[0010] Step S4, for the first image, each feature matching point is eroded according to a preset erosion direction, and the remaining region after erosion is the topmost tobacco leaf obtained by segmentation.

[0011] Preferably, the step S2 specifically comprises:

[0012] Step S21, constructing a plurality of similar feature points according to the first image and the second image, each of the feature points comprising a first feature point in the first image and a corresponding second feature point in the second image;

[0013] Step S22, extracting a first descriptor of each of the first feature points in the first image in a plurality of dimensional directions and a second descriptor of each of the second feature points in the second image in the plurality of dimensional directions, respectively;

[0014] Step S23, performing similarity calculation according to the first descriptor and the second descriptor corresponding to each of the feature points, and then determining the feature matching point according to the similarity.

[0015] Preferably, in the step S23, the feature matching point is determined by using the following formula:

[0016] Pair(Pi, Pi') = max(Near(Sim(Pi, Pi')))

[0017] wherein Pi represents the i-th first feature point; Pi' represents the corresponding i-th second feature point; Sim(Pi, Pi') represents the similarity of the first descriptor and the second descriptor corresponding to the i-th feature point pair; Near(Sim(Pi, Pi')) represents the feature point with the similarity in a preset region; Pair(Pi, Pi') represents the determined feature matching point.

[0018] Preferably, the preset region is a potential point region with a size of M*M centered on each of the feature point pairs.

[0019] Preferably, the step S3 specifically comprises:

[0020] removing the feature matching point with a similarity lower than a preset similarity threshold; and / or

[0021] removing the feature matching point with a matching distance of the first feature point and the second feature point exceeding a preset distance threshold.

[0022] Preferably, the first descriptor and the second descriptor are response values of each of the feature points in a plurality of preset dimensional directions.

[0023] Preferably, the calculation method of the response value is:

[0024] obtaining a neighboring feature point of each of the feature points in all of the preset dimensional directions at a preset distance;

[0025] Calculate pixel absolute difference values of the feature points and the adjacent feature points in each of the preset dimension directions, to obtain the response values of the feature points in each of the preset dimension directions.

[0026] Preferably, the step S4 specifically comprises:

[0027] Step S41, divide the first image into several regions, and each region corresponds to a preset erosion direction.

[0028] Step S42, according to the preset erosion direction corresponding to the region where each feature matching point is located, erosion is performed to obtain the uppermost tobacco leaf.

[0029] The application also provides a system for segmenting the uppermost tobacco leaf based on dense feature point matching, which is used to implement the method for segmenting the uppermost tobacco leaf based on dense feature point matching.

[0030] An image acquisition module is configured to acquire a first image before the uppermost tobacco leaf is grabbed and a second image after the uppermost tobacco leaf is grabbed.

[0031] A dense matching module is connected to the image acquisition module and is configured to perform dense feature point matching on the tobacco leaf regions of the first image and the second image to obtain feature matching points.

[0032] An abnormality processing module is connected to the dense matching module and is configured to remove abnormal feature matching points.

[0033] An analysis module is connected to the abnormality processing module and is configured to, for the first image, perform erosion on each feature matching point according to a preset erosion direction, and the remaining region after the erosion is the uppermost tobacco leaf obtained by segmentation.

[0034] The application has the following advantages or beneficial effects:

[0035] The application creatively proposes a method for segmenting the uppermost tobacco leaf from a stack of superimposed tobacco leaves by using dense feature point matching, thereby realizing the grading processing of the uppermost tobacco leaf. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 For the preferred embodiment of the application, a flowchart of the method for segmenting the uppermost tobacco leaf based on dense feature point matching is shown.

[0037] Figure 2 For the preferred embodiment of the application, a flowchart of the implementation of step S2 is shown.

[0038] Figure 3For the preferred embodiment of the present application, the flow chart of the implementation of the calculation method of the response value is shown in the figure;

[0039] Figure 4 For the preferred embodiment of the present application, the schematic diagram of the first image and the second image after step S2 is implemented is shown in the figure;

[0040] Figure 5 For the preferred embodiment of the present application, the schematic diagram of the first image after the exception processing in step S3 is shown in the figure;

[0041] Figure 6 For the preferred embodiment of the present application, the schematic diagram of the implementation of step S4 is shown in the figure;

[0042] Figure 7 For the preferred embodiment of the present application, the schematic diagram of the implementation of the corrosion direction is shown in the figure;

[0043] Figure 8 For the preferred embodiment of the present application, the structure block diagram of the topmost tobacco leaf segmentation system based on dense feature point matching is shown in the figure. DETAILED DESCRIPTION

[0044] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0045] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0046] The present application will be further described below with reference to the drawings and specific embodiments, but is not limited by the present application.

[0047] In the preferred embodiment of the present application, in order to solve the above problems existing in the prior art, a topmost tobacco leaf segmentation method based on dense feature point matching is provided, which belongs to the technical field of tobacco leaf grading, as shown in the figure, comprising: Figure 1

[0048] Step S1, acquiring a first image and a second image containing a plurality of stacked tobacco leaves, the first image being an image before the topmost tobacco leaf is grabbed, and the second image being an image after the topmost tobacco leaf is grabbed;

[0049] Specifically, before the topmost tobacco leaf is grabbed by the grabbing mechanism, the first image is obtained by shooting through the image acquisition module 1, and then the second image is obtained by shooting through the image acquisition module 1 after the topmost tobacco leaf is grabbed by the grabbing mechanism. The image acquisition module 1 can be a shooting device such as a common camera or a video camera.​

[0050] Then, the first image and the second image are sent to the dense matching module 2, and further, before the dense feature point matching is performed, the following is further included:

[0051] The tobacco leaf regions in the first image and the second image are extracted respectively.

[0052] Further, the method for extracting the tobacco leaf region is as follows: the non-tobacco leaf region is blackened according to the gray value of the original image, and specifically:

[0053] It is judged whether the image gray value is greater than a preset reference value, and the preset reference value can be set according to actual needs:

[0054] If the image gray value is greater than the preset reference value, the gray value thereof is set to 0;

[0055] If the image gray value is less than the preset reference value, no processing is performed.

[0056] Step S2, the dense feature point matching is performed on the tobacco leaf regions of the first image and the second image, and the feature matching points are obtained.

[0057] Step S3, the abnormal feature matching points are removed.

[0058] Step S4, for the first image, each feature matching point is eroded according to a preset erosion direction, and the remaining region after the erosion is the topmost tobacco leaf obtained by the segmentation.

[0059] Specifically, in the embodiment, the method of using the dense feature point matching is creatively proposed to segment the topmost tobacco leaf from a stack of superimposed tobacco leaves, so as to realize the grading processing of the topmost tobacco leaf, and the segmentation method has high robustness to changes in the external environment.

[0060] As a preferred embodiment, wherein, as shown in Figure 2 Step S2 specifically includes:

[0061] Step S21, a plurality of similar feature points are constructed according to the first image and the second image, and each pair of feature points includes a first feature point in the first image and a corresponding second feature point in the second image;

[0062] Step S22, a first descriptor of each first feature point in the first image in a multidimensional direction and a second descriptor of each second feature point in the second image in the multidimensional direction are extracted respectively;

[0063] Step S23, the similarity is calculated according to the first descriptor and the second descriptor corresponding to each pair of feature points, and then the feature matching points are determined according to the similarity.

[0064] Further, due to the high similarity of the tobacco leaves, the color change of different tobacco leaves in the two images before and after the tobacco leaves are picked will only fluctuate in a small range, and the position change of the tobacco leaves to be picked is small. The existing dense feature point matching is mainly global matching, which does not fully utilize the small position change of the tobacco leaves to be picked, and is prone to matching errors. In the embodiment, after the descriptor is constructed, the nearest neighbor search is performed on the corresponding feature points to limit the global matching area, which significantly improves the matching quality and avoids the occurrence of false matching.

[0065] As a preferred embodiment, the first descriptor and the second descriptor are respectively the response values of each corresponding feature point in a plurality of preset dimension directions.

[0066] Specifically, due to the high similarity of the tobacco leaves, the existing dense feature points are mainly based on high-contrast images in nature, which is prone to low matching confidence. In the embodiment, when constructing the descriptor, multi-dimensional directions are used to calculate the descriptor, so that the descriptor contains more feature information, thereby significantly improving the matching quality. As an example but not limitation, the preset dimension direction is preferably eight dimensions, specifically: taking a direction (such as directly above) as a reference, the feature information of each feature point in 0, 45, 90, 135, 180, 225, 270, and 315 degrees.

[0067] As a preferred embodiment, the calculation method of the response value is as shown in Figure 3

[0068] A1, obtaining the adjacent feature points of each feature point in all preset dimension directions at a preset distance;

[0069] A2, calculating the pixel absolute difference value of the feature point and each adjacent feature point in the preset dimension direction to obtain the response value of the feature point in each preset dimension direction.

[0070] Specifically, the dense feature point matching process is as follows: first, based on the two images before and after picking the uppermost tobacco leaves, similar feature points are constructed, and the construction process of the similar feature points can be realized by using the existing dense feature point matching algorithm; then, the descriptor of the feature point is calculated; and then, the feature matching points are calculated according to the similarity of each dimension of the descriptor. The feature matching points obtained by the above calculation are displayed in the corresponding images, as shown in Figure 4

[0071] For the descriptor of each feature point in each image, it specifically includes eight dimensions of feature information, which is denoted as:

[0072] Di = [V0, V​​45 ,V 90 ,V 135 ,V 180 ,V 225 ,V 270 ,V 315 ] (1)

[0073] wherein, V0, V 45 ,V 90 ,V 135 ,V 180 ,V 225 ,V 270 ,V 315 respectively represent the feature information of each feature point in 0, 45, 90, 135, 180, 225, 270, 315 degree directions; Di represents the descriptor of the i-th feature point.

[0074] The calculation formula of each single dimension feature information is as follows:

[0075] V j = abs(Val(Pi)-val(Pi j ) (2)

[0076] wherein, Val(Pi) represents the pixel value of the i-th feature point; val(Pi) j represents the adjacent feature point with the i-th feature point in j dimension direction at a preset distance, which is preferably 2; V j represents the feature information of the j-th dimension.

[0077] Then, the similarity of the feature points is calculated, and the calculation of the similarity adopts the normalized square difference similarity measurement method, as follows formula (3):

[0078] Sim(Pi, Pi') = sum(Di j -Di j ') / sqrt(sum(Di j *Di j ) * sum(Di j ' * Di j '))

[0079] wherein, j = 0, 45, 90, 135, 180, 225, 270, 315; (3)

[0080] wherein, Pi represents the i-th first feature point in the first image; Pi' represents the corresponding i-th second feature point; Di j represents the feature information of the j-th dimension direction in the descriptor of the i-th feature point in the first image; Di jDij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; sum(Di j Dij represents the feature information in the jth dimension direction of the descriptor of the ith feature point in the second image; Sim(Pi, Pi') represents the similarity of the first descriptor and the second descriptor corresponding to the ith pair of feature points.

[0081] As a preferred embodiment, in step S23, the feature matching point is determined by using the following formula:

[0082] Pair(Pi, Pi') = max(Near(Sim(Pi, Pi')))

[0083] wherein, Pi represents the ith first feature point; Pi' represents the corresponding ith second feature point; Sim(Pi, Pi') represents the similarity of the first descriptor and the second descriptor corresponding to the ith pair of feature points; Near(Sim(Pi, Pi')) represents the feature points with similarity in the preset region; Pair(Pi, Pi') represents the determined feature matching point.

[0084] As a preferred embodiment, the preset region is a potential point region with a size of M*M centered on each pair of feature points, and M is preferably 5.

[0085] As a preferred embodiment, in step S3, the feature matching point is subjected to an abnormality processing, which comprises:

[0086] removing the feature matching point with a similarity lower than a preset similarity threshold.

[0087] As a preferred embodiment, in step S3, the feature matching point is subjected to an abnormality processing, which further comprises:

[0088] removing the feature matching point with a matching distance between the first feature point and the second feature point exceeding a preset distance threshold.

[0089] The preset similarity threshold and the preset distance threshold can be set according to actual application, which is not limited herein.

[0090] Further, the matching distance between the first feature point and the second feature point is calculated as follows:

[0091] dist(Pi, Pi') = sqrt((pi_x - pi_x') 2 +(pi_y - pi_y') 2 ) (5)

[0092] wherein, pi_x, pi_y represent the pixel coordinates of the i-th feature point in the first image; pi_x', pi_y' represent the pixel coordinates of the i-th feature point in the second image; dist(Pi, Pi') represents the matching distance of the first feature point and the second feature point in the similar feature points.

[0093] Specifically, in the embodiment, the feature matching point abnormality processing can also combine the above two abnormality processing schemes, that is, first, remove the feature points with low similarity according to the similarity, and then remove the feature points with far matching distance according to the distance of the feature points in the two images, as shown in the effect diagram after removing the abnormal feature matching points. Figure 5

[0094] As a preferred embodiment, as shown in Figure 6 , step S4 specifically includes:

[0095] Step S41, dividing the first image into a plurality of regions, each region corresponding to a preset erosion direction;

[0096] Step S42, according to the preset erosion direction corresponding to the region where each feature matching point is located, erosion is performed to obtain the uppermost tobacco.

[0097] Specifically, in the embodiment, each image is composed of a plurality of regions, each region corresponds to a preset erosion direction, and the preset erosion directions corresponding to different regions are different. For each feature matching point, first determine the region where each feature matching point is located in the image; then according to the preset erosion direction corresponding to the determined region, erosion is performed to the edge direction of the image respectively, and the gray value of the eroded region is set to 0. As an example but not limitation, the specific erosion direction is as shown in Figure 7 .

[0098] The application also provides a system for segmenting the uppermost tobacco based on dense feature point matching, which is used to implement the method for segmenting the uppermost tobacco based on dense feature point matching as described above, as shown in Figure 8 , comprising:

[0099] An image acquisition module 1 is used to acquire the first image before the uppermost tobacco is grabbed and the second image after the uppermost tobacco is grabbed;

[0100] A dense matching module 2 is connected to the image acquisition module 1 and is used to perform dense feature point matching on the tobacco region of the first image and the second image to obtain feature matching points.​

[0101] an exception processing module 3 connected to the dense matching module 2, for removing abnormal feature matching points;

[0102] an analysis module 4 connected to the exception processing module 3, for, for each feature matching point, according to a preset erosion direction, eroding the first image, and the remaining region after the erosion being the topmost tobacco leaf obtained by the segmentation.

[0103] Further, the mechanical chassis (not shown in the figure) is further included, and the stacked tobacco leaves are placed on the mechanical chassis;

[0104] a grabbing mechanism (not shown in the figure), for grabbing the topmost tobacco leaf.

[0105] Specifically, in the embodiment, before the grabbing mechanism is used to grab the topmost tobacco leaf, the first image is obtained by the image acquisition module 1, and then the second image is obtained by the image acquisition module 1 after the grabbing mechanism grabs the topmost tobacco leaf, the image acquisition module 1 can be a common camera, a video camera or the like; the first image and the second image are sent to the dense matching module 2 for dense feature point matching, and then the exception processing is performed, finally, the analysis processing is performed, that is, according to the determined feature matching point, the image edge is eroded, and the remaining region after the erosion is the topmost tobacco leaf to be segmented.

[0106] The above only describes the preferred embodiments of the present application, and does not limit the implementation and protection scope of the present application. It should be realized by those skilled in the art that any equivalent replacement and obvious changes made according to the content of the present application should be included in the protection scope of the present application.

Claims

1. A method for topmost tobacco leaf segmentation based on dense feature point matching, characterized in that, The method comprises the following steps: S1, acquiring a first image and a second image containing a plurality of layers of tobacco superimposed, the first image being an image before the topmost layer of tobacco is grabbed, and the second image being an image after the topmost layer of tobacco is grabbed; S2, performing dense feature point matching on the tobacco area of the first image and the second image to obtain feature matching points; S3, removing abnormal feature matching points; S4, for the first image, each feature matching point is eroded according to a preset erosion direction, and the remaining area after erosion is the topmost layer of tobacco obtained by segmentation; The step S2 specifically comprises: S21, constructing a plurality of similar feature points according to the first image and the second image, each pair of feature points comprising a first feature point in the first image and a corresponding second feature point in the second image; S22, extracting a first descriptor of each first feature point in the first image in a plurality of dimensional directions, and a second descriptor of each second feature point in the second image in a plurality of dimensional directions; S23, performing similarity calculation on the first descriptor and the second descriptor corresponding to each pair of feature points, and then determining the feature matching points according to the similarity; In the step S23, the feature matching points are determined by the following formula: Pair(Pi, Pi') = max(Near(Sim(Pi, Pi'))) Wherein, Pi represents the i-th first feature point; Pi' represents the corresponding i-th second feature point; Sim(Pi, Pi') represents the similarity of the first descriptor and the second descriptor corresponding to the i-th pair of feature points; Near(Sim(Pi, Pi')) represents the feature points in the preset region; Pair(Pi, Pi') represents the determined feature matching points; The first descriptor and the second descriptor are respectively the response value of each feature point in a plurality of preset dimensional directions; The calculation method of the response value is: Obtaining the adjacent feature points of each feature point at a preset distance in all preset dimensional directions; Calculating the pixel absolute difference value of the feature point and each adjacent feature point in the preset dimensional direction to obtain the response value of the feature point in each preset dimensional direction. 2.The uppermost tobacco leaf segmentation method based on dense feature point matching according to claim 1, wherein, The preset region is a potential point region with a size of M*M centered on each pair of feature points. 3.The uppermost tobacco leaf segmentation method based on dense feature point matching according to claim 1, wherein, The step S3 specifically comprises: Removing the feature matching points with a similarity lower than a preset similarity threshold; and / or Removing the feature matching points with a matching distance of the first feature point and the second feature point exceeding a preset distance threshold. 4.The uppermost tobacco leaf segmentation method based on dense feature point matching according to claim 1, wherein, The step S4 specifically comprises: S41, dividing the first image into a plurality of regions, each region corresponding to a preset erosion direction; S42, eroding according to the preset erosion direction corresponding to the region where each feature matching point is located to obtain the topmost layer of tobacco.

5. A topmost tobacco leaf segmentation system based on dense feature point matching, for implementing the topmost tobacco leaf segmentation method based on dense feature point matching according to any one of claims 1-4, characterized in that, The method comprises the following steps: An image acquisition module is configured to acquire a first image before the uppermost tobacco leaf is picked up and a second image after the uppermost tobacco leaf is picked up; A dense matching module is connected to the image acquisition module and configured to perform dense feature point matching on the tobacco leaf area of the first image and the second image to obtain feature matching points; An abnormality processing module is connected to the dense matching module and configured to remove abnormal feature matching points; An analysis module is connected to the abnormality processing module and configured to, for the first image, perform corrosion on each feature matching point according to a preset corrosion direction, and the remaining area after corrosion is the uppermost tobacco leaf obtained by segmentation. The dense matching module is configured to construct a plurality of similar feature points according to the first image and the second image, each pair of feature points including a first feature point in the first image and a corresponding second feature point in the second image; extract a first descriptor of each first feature point in the first image in a plurality of dimensional directions and a second descriptor of each second feature point in the second image in a plurality of dimensional directions; perform similarity calculation on the first descriptor and the second descriptor corresponding to each pair of feature points, and then determine the feature matching points according to the similarity; and determine the feature matching points by using the following formula: Pair(Pi, Pi') = max(Near(Sim(Pi, Pi'))) wherein Pi represents the i-th first feature point; Pi' represents the corresponding i-th second feature point; Sim(Pi, Pi') represents the similarity of the first descriptor and the second descriptor corresponding to the i-th pair of feature points; Near(Sim(Pi, Pi')) represents the feature points in a preset region; Pair(Pi, Pi') represents the determined feature matching points; The first descriptor and the second descriptor are response values of each feature point in a plurality of preset dimensional directions; and the response values are calculated by: acquiring adjacent feature points of each feature point in all preset dimensional directions at a preset distance; and calculating pixel absolute difference values of the feature point and the adjacent feature points in each preset dimensional direction to obtain the response values of the feature point in each preset dimensional direction.

Images