Tobacco leaf length prediction method and system
The tobacco leaf image is processed through the image semantic segmentation model, and its length is calculated according to the shape of the tobacco leaf, which solves the problem of difficulty in accurately measuring the length of different shapes of tobacco leaves in the existing technology and improves the accuracy of tobacco leaf classification.
Patent Information
- Application Number
- CN202411468386.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-01-24
AI Technical Summary
Existing tobacco leaf length measurement technology is difficult to accurately measure the length of tobacco leaf under different shapes (such as triangles, S-bending, etc.), resulting in low accuracy of tobacco leaf grading.
By constructing and training the semantic segmentation model of the image, the original tobacco leaf image is obtained and the segmentation processing is performed.
The accurate length measurement of different shapes of tobacco leaves has been achieved, and the accuracy and efficiency of the tobacco classification are improved.
Smart Images
Figure CN119360022A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of tobacco leaf detection, and in particular to a tobacco leaf length prediction method and system. Background Art
[0002] The tobacco industry plays an important role in the development of my country's economy. In the purchase of tobacco leaves, tobacco leaf grading not only determines the quality of flue-cured tobacco, but also determines the purchase price of tobacco leaves, which in turn affects the economic index of the tobacco industry. Therefore, how to accurately control tobacco leaf grading is particularly important.
[0003] At present, the existing tobacco leaf grading technology in my country mainly relies on manual empirical inspection. The accuracy of this method is affected by factors such as strong human subjective arbitrariness, resulting in uneven accuracy of tobacco leaf grading. With the continuous development of modern artificial intelligence technology, the use of artificial intelligence computer vision technology for tobacco leaf grading has developed rapidly in recent years, which has improved the accuracy and efficiency of tobacco leaf grading.
[0004] The parameters that affect the grade of tobacco leaves include length, thickness, color, oil content, etc. The length of tobacco leaves is usually between 25 cm and 45 cm, with 5 cm as the range of change in tobacco leaf grade. Therefore, accurate measurement of tobacco leaf length is the core parameter for grade classification. After the tobacco leaves are picked, they go through a series of operations such as baking and transportation, which can easily cause the tobacco leaves to form a curved shape. The existing tobacco leaf length measurement technology mainly measures tobacco leaves in a vertical shape. However, there is little research on the accurate measurement of tobacco leaves in irregular shapes such as triangles and S-curves.
[0005] Therefore, providing a tobacco leaf length prediction method and system for accurate measurement of tobacco leaves of different shapes is an urgent problem to be solved by those skilled in the art. Summary of the invention
[0006] The object of the present invention is to provide a tobacco leaf length prediction method, which has clear logic, is safe, effective, reliable and easy to operate, can adopt different processing and calculation methods for different forms of tobacco leaves, and can efficiently and accurately predict the length of tobacco leaves in various forms.
[0007] A tobacco leaf length prediction method comprises the following steps: Build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; Obtaining an original tobacco leaf image, and inputting the image into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; Processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; Determining the shape of the tobacco leaves according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio; Calculate and obtain the tobacco leaf length corresponding to the tobacco leaf morphology.
[0008] Preferably, the step of constructing and training the image semantic segmentation model to obtain the trained image semantic segmentation model comprises the following steps: Construct an initial image semantic segmentation model consisting of convolutional layers, activation functions, and pooling layers, using sticky connections; The iterative training enables the initial image semantic segmentation model to converge, thereby obtaining a trained image semantic segmentation model.
[0009] Preferably, the step of acquiring the original tobacco leaf image and inputting it into the trained image semantic segmentation model to obtain the segmented tobacco leaf image comprises the following steps: Acquiring the original tobacco leaf image by an industrial camera installed at a fixed position; The trained image semantic segmentation model segments the input original tobacco leaf image to extract tobacco leaf regions as segmented tobacco leaf images.
[0010] Preferably, the step of processing the segmented tobacco leaf image according to an image processing tool to obtain the actual value of the tobacco leaf aspect ratio comprises the following steps: The segmented tobacco leaf image is processed by grayscale and binarization in sequence by the image processing tool to obtain a tobacco leaf image matrix; Extracting tobacco leaf contours according to the image processing tool and the tobacco leaf image matrix; According to the tobacco leaf contour and the minimum circumscribed rectangular frame method, the endpoint coordinate values of the minimum circumscribed rectangular frame are obtained; Repeat the above steps to obtain the actual value of the tobacco leaf aspect ratio according to the endpoint coordinate values of the multiple minimum circumscribed rectangular frames and the preset formula.
[0011] Preferably, determining the tobacco leaf morphology according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio comprises the following steps: Determining whether the actual value of the tobacco leaf length-to-width ratio is not greater than the tobacco leaf length-to-width ratio threshold; If yes, the tobacco leaf is defined to be in a vertical position; If not, the tobacco leaf is defined as being in an irregular shape.
[0012] Preferably, the calculating and obtaining the tobacco leaf length corresponding to the tobacco leaf morphology comprises the following steps: When the tobacco leaf is in a vertical state, the first tobacco leaf length is calculated and obtained according to the coordinate values of the endpoints of the minimum circumscribed rectangular frame.
[0013] Preferably, the step of calculating and obtaining the tobacco leaf length corresponding to the tobacco leaf shape further includes the following steps: When the tobacco leaf is in an irregular shape, extracting extreme points in different directions from the contour of the tobacco leaf; Connecting the extreme points in different directions of the tobacco leaf contour in sequence to obtain an extreme point straight line; According to the Bezier curve algorithm, the extreme point straight line is converted into an extreme point curve; Calculating the total length of the extreme point curve according to a combined weight method based on Euclidean distance; The second tobacco leaf length is obtained according to the parameters of the industrial camera and the total length of the extreme point curve.
[0014] Preferably, the step of extracting extreme points in different directions from the tobacco leaf contour comprises the following steps: The zero-crossing point operator of the second-order derivative of the image intensity is introduced into the preset gradient formula to calculate the gradient value of each coordinate point, and the contour of the tobacco leaf is extracted according to the gradient difference; A pixel query is performed on the tobacco leaf contour to obtain extreme points in different directions.
[0015] Preferably, the total length of the extreme point curve is calculated according to the combined weight method based on Euclidean distance, comprising the following steps: Define each round coordinate points, forming Line segments, the coordinate value cluster is ; Then the Euclidean distance formula for each line segment is: ; The total length of the curve is given by: .
[0016] A tobacco leaf length prediction system, comprising: A model building module is used to build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; An image segmentation module is used to obtain an original tobacco leaf image and input it into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; A tobacco leaf aspect ratio module, used for processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; A tobacco leaf morphology module, used to determine the tobacco leaf morphology according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio; The tobacco leaf length module is used to calculate and obtain the tobacco leaf length corresponding to the tobacco leaf morphology.
[0017] The tobacco leaf length prediction method provided by the present invention constructs an image semantic segmentation model and trains it to obtain a trained image semantic segmentation model; inputs the collected original tobacco leaf image into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; processes the segmented tobacco leaf image by an image processing tool to obtain an actual value of the tobacco leaf length-to-width ratio; presets a tobacco leaf length-to-width ratio threshold in advance, compares it with the actual value of the tobacco leaf length-to-width ratio, and determines the tobacco leaf morphology according to the comparison result; and calculates the corresponding tobacco leaf length according to different tobacco leaf morphologies.
[0018] The present invention determines different forms of tobacco leaves and selects corresponding processing and calculation methods according to the different forms to obtain the corresponding predicted tobacco leaf length. Compared with the prior art, the present invention designs a corresponding tobacco leaf length processing and calculation method for the special forms of tobacco leaves, which can effectively improve the universality and accuracy of tobacco leaf length prediction.
[0019] The present invention also provides a tobacco leaf length prediction system. Since the prediction system and the prediction method belong to the same technical field, solve the same technical problems, and should have the same beneficial effects, they will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0021] Figure 1 A flowchart of a tobacco leaf length prediction method provided by an embodiment of the present invention; Figure 2 A flowchart of step S1 provided in an embodiment of the present invention; Figure 3 A schematic diagram of the Unet++ network architecture provided by an embodiment of the present invention; Figure 4 A flowchart of step S2 provided in an embodiment of the present invention; Figure 5 A flowchart of step S3 provided in an embodiment of the present invention; Figure 6 A flowchart of step S5 provided in an embodiment of the present invention; Figure 7 A flowchart of step D1 provided in an embodiment of the present invention; Figure 8 A quadratic Bezier curve analysis diagram provided by an embodiment of the present invention; Fig. 9A schematic diagram of the structure of a tobacco leaf length prediction system provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] The embodiments of the present invention are written in a progressive manner.
[0024] The embodiment of the present invention provides a tobacco leaf length prediction method and system, which mainly solves the technical problem in the prior art that the length prediction result of the existing tobacco leaf is inaccurate due to the single state.
[0025] like Figure 1 As shown, a tobacco leaf length prediction method comprises the following steps: S1. Build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; S2. Obtaining the original tobacco leaf image, and inputting it into the trained image semantic segmentation model to obtain the segmented tobacco leaf image; S3. Processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; S4. Determine the shape of the tobacco leaf according to the actual value of the tobacco leaf length-width ratio and the preset threshold value of the tobacco leaf length-width ratio; S5. Calculate and obtain the tobacco leaf length corresponding to the tobacco leaf shape.
[0026] In step S1, an initial image semantic segmentation model is constructed, and the image semantic segmentation model is trained to obtain a trained image semantic segmentation model; In this embodiment, in order to avoid the influence of external factors in the image on the extraction of tobacco leaf images, the deep learning Unet image semantic segmentation convolutional neural network is used as the algorithm framework, and the concept of Densenet fully connected residual neural network is added on its basis to obtain the initial semantic segmentation model. The initial semantic segmentation model is trained by inputting the sample set to obtain the trained image semantic segmentation model; In step S2, after the original tobacco leaf image is captured by an industrial camera, the original tobacco leaf image is input into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; In step S3, a corresponding image processing tool is set to process the segmented tobacco leaf image through the image processing tool, and after the processing is completed, the actual value of the aspect ratio of the tobacco leaf is obtained; In step S4, a threshold value of the tobacco leaf length-to-width ratio is preset in advance, compared with the actual value of the tobacco leaf length-to-width ratio, and the tobacco leaf morphology is determined according to the comparison result; In step S5, different processing and calculation methods are set according to different tobacco leaf shapes to obtain the tobacco leaf length corresponding to the tobacco leaf shape.
[0027] like Figure 2 As shown, preferably, step S1 includes the following steps: A1. Construct an initial image semantic segmentation model consisting of convolutional layers, activation functions, and pooling layers, using sticky connections; A2. Iterative training makes the initial image semantic segmentation model converge to obtain the trained image semantic segmentation model.
[0028] In step A1 to step A2, the deep learning Unet image semantic segmentation convolutional neural network is used as the algorithm framework, and the concept of Densenet fully connected residual neural network (i.e., sticky connection method) is added to obtain the initial image semantic segmentation model. 80% of the pre-collected data set (consisting of multiple original tobacco leaf images) is divided into a training set and 20% is divided into a test set, where the training set is used for training and the test set is used for accuracy detection. Use Transform to convert the image into data of size (300, 300), and use Torch to convert the data into Tensor format. Train 200 times to obtain the training weight with the highest accuracy, and edit the prediction model for adjustment to obtain the trained image semantic segmentation model.
[0029] In this embodiment, the feature extraction module of the Unet image semantic segmentation network architecture consists of a 3*3 convolutional layer, a Relu activation function, and a pooling layer. The convolutional layer converts the image into a matrix through the Torch module in Pytorch, and gradually scans the convolution kernel in the two-dimensional data. As the convolution kernel slides and calculates the product of the matrix and the scanned data matrix, the results are summarized to obtain a new matrix, which plays the role of reducing the dimension and extracting feature points. In order to prevent the information of the convolutional layer from being fully learned, the Densenet residual fully connected convolutional neural network can not only learn the convolutional layer information transmitted by the previous layer, but also add the convolutional layer information of each previous layer to the new convolutional module; The initial image semantic segmentation model Unet++ inherits the structure of the Uent image semantic segmentation deep learning network architecture, and absorbs the sticky connection method of the Densenet residual fully connected neural network architecture, so that each layer can receive the information of each previous convolutional layer, and each module before and after interacts with each other. Each module can see each other's information and is familiar with each other. Finally, the information is shared with the last layer, so that the segmentation effect is better.
[0030] like Figure 3 As shown in the figure, Backbone is the backbone network of the Unet++ image semantic segmentation network architecture. Its main function is to extract feature information and generate feature maps. Down-sampling is called upsampling, which is often used to restore the spatial size of the feature map to the original input size, or to increase the resolution of the feature map. Common upsampling operations include deconvolution, which upsamples by learning a reversible convolution kernel, filling and interpolating the feature map, thereby increasing the size of the feature map.
[0031] Up-sampling is called upsampling, and upsampling is usually called pooling. Its function is to reduce the spatial size of the feature map. The most common pooling operation is max pooling, which divides the original feature map into small non-overlapping areas, and then selects the maximum value in each area as the sampling point, thereby reducing the width and height of the feature map. The pooling operation helps to extract the translation invariance of features, while reducing the size of the feature map, reducing the number of parameters and calculations, and alleviating overfitting.
[0032] Skip Connection is called Residual Connection. The idea of residual connection is to connect the input directly to the output in certain layers of the network to allow information to jump between different layers. This connection is usually implemented through an addition operation to merge the input and output together. Residual connection can solve the gradient disappearance problem in deep neural networks and accelerate the training process. Deep neural networks usually contain many layers. When the gradient signal is back-propagated, the gradient may become very small, making the network difficult to train. Skip connection allows the gradient to be back-propagated more easily by passing the input information directly to subsequent layers. This direct transfer helps to maintain the stability of the gradient and avoid the gradient disappearance.
[0033] like Figure 4 As shown, preferably, step S2 includes the following steps: B1. Obtain the original tobacco leaf image through an industrial camera installed at a fixed position; B2. The trained image semantic segmentation model segments the input original tobacco leaf image to extract the tobacco leaf region as the segmented tobacco leaf image.
[0034] In steps B1 to B2, an industrial camera with a fixed shooting position is provided in the tobacco leaf grading equipment, the height between the industrial camera and the conveyor belt remains unchanged, and the size of the captured image remains unchanged; after the original tobacco leaf image is obtained by the industrial camera, it is input into the trained image semantic segmentation model, and the model segments the original tobacco leaf image and extracts the tobacco leaf area as the segmented tobacco leaf image.
[0035] In this embodiment, the tobacco leaf part in the original tobacco leaf image is segmented along the edge of the tobacco leaf through the trained image semantic segmentation model, the tobacco leaf area is retained, and other areas are filled with a black image. After the processing is completed, a segmented tobacco leaf image is obtained.
[0036] like Figure 5 As shown, preferably, step S3 includes the following steps: C1. grayscale and binarize the segmented tobacco leaf image in sequence using image processing tools to obtain a tobacco leaf image matrix; C2. extracting the outline of tobacco leaves according to the image processing tool and the tobacco leaf image matrix; C3. Obtain the endpoint coordinates of the minimum bounding rectangle according to the tobacco leaf contour and the minimum bounding rectangle method; C4. Repeat the above steps to obtain the actual value of the tobacco leaf aspect ratio according to the endpoint coordinate values of multiple minimum circumscribed rectangular frames and a preset formula.
[0037] In step C1 to step C4, an image processing tool is installed to grayscale and binarize the segmented tobacco leaf image in sequence to obtain a tobacco leaf image matrix; the tobacco leaf contour in the tobacco leaf image is extracted through the tobacco leaf image matrix and the image processing tool; the minimum circumscribed rectangular frame method and the tobacco leaf contour are used to obtain the endpoint coordinate values of the minimum circumscribed rectangular frame of the tobacco leaf contour; the above steps are repeated to obtain the endpoint coordinate values of multiple minimum circumscribed rectangular frames, and the actual value of the tobacco leaf aspect ratio is calculated according to a preset formula.
[0038] In this embodiment, the installation tool: OpenCV image processing package, using cv2.imread() function to read the segmented tobacco leaf high-definition image, converting the RGB format of the tobacco leaf image into a matrix format for easy computer reading; using the findContours() function in OpenCV to find the contour in the image, where the cv2.RETR_EXTERNAL function only detects the outermost contour of the tobacco leaf, and the cv2.CHAIN_APPROX_SIMPLE function indicates the compression of the elements in the horizontal, vertical and diagonal directions, and retains the coordinate values of the parameters; secondly, using the cv2.contourArea() in the max() function to find the contour with the largest area, the function can filter out the error caused by the residue of the tobacco leaf; then, using the cv2.minAreaRect() and cv2.boxpoints() functions to calculate the information of the minimum bounding rectangle; finally, using the cv2.cvtColor() function to obtain the coordinate value of the minimum bounding rectangle. The length of the minimum bounding rectangle is the length index of the tobacco leaf.
[0039] The four coordinate values are , , , , where the four coordinate values are lower left, upper left, upper right, and lower right. The length is L, the width is M, is the length-to-width ratio, and the preset formula for the actual value of the tobacco leaf length-to-width ratio is as follows:
[0040]
[0041] .
[0042] Preferably, step S4 comprises the following steps: Determine whether the actual value of the tobacco leaf length-to-width ratio is not greater than the tobacco leaf length-to-width ratio threshold; If yes, the tobacco leaf is defined to be in a vertical position; If not, the tobacco leaf is defined as being in an irregular shape.
[0043] In step S4, by comparing the actual value of the tobacco leaf's length-to-width ratio with the tobacco leaf's length-to-width ratio threshold, when the actual value of the tobacco leaf's length-to-width ratio is less than or equal to the tobacco leaf's length-to-width ratio threshold, the tobacco leaf is in a vertical state, and when the actual value of the tobacco leaf's length-to-width ratio is greater than the tobacco leaf's length-to-width ratio threshold, the tobacco leaf is in an irregular shape; In this embodiment, the toughness of tobacco leaves is reduced after being baked, and they are easily deformed into irregular shapes such as bends, triangles, and S shapes during the processes of artificial bundling, transportation, and handling. When tobacco leaves are in irregular shapes, the method of measuring the length of the minimum external frame of the tobacco leaves will bring about large errors.
[0044] According to the above formula, the four coordinate values are calculated separately using the minimum bounding box method.
[0045] The aspect ratio dataset of tobacco leaves is defined as: , where n is the number of samples, and the actual threshold of the length-to-width ratio of tobacco leaves (here the mean is chosen as the threshold) is:
[0046] If the length-to-width ratio of the tobacco leaf is greater than , indicating that the tobacco leaf is in a curved shape; if the length-to-width ratio of the tobacco leaf is less than or equal to This indicates that the tobacco leaves are in a vertical position.
[0047] Preferably, step S5 comprises the following steps: When the tobacco leaf is in a vertical state, the length of the first tobacco leaf is calculated based on the coordinate values of the endpoints of the minimum circumscribed rectangular frame.
[0048] In this embodiment, when the tobacco leaf is in a vertical state, the length of the minimum circumscribed rectangular frame is the length of the first tobacco leaf, that is, the length L in the above formula.
[0049] like Figure 6 As shown, preferably, step S5 further includes the following steps: D1. When the tobacco leaf is in an irregular shape, extract the extreme points in different directions of the tobacco leaf contour; D2. Connect the extreme points in different directions of the tobacco leaf contour in sequence to obtain an extreme point straight line; D3. According to the Bezier curve algorithm, the extreme point straight line is converted into an extreme point curve; D4. Calculate the total length of the extreme point curve according to the combined weight method based on Euclidean distance; D5. Obtain the second tobacco leaf length according to the parameters of the industrial camera and the total length of the extreme point curve.
[0050] In step D1 to step D5, the core concept of the length measurement method of the irregular shape of tobacco leaves is to extract each pixel point in the grayscale image, extract the contour by calculating the gradient difference, and then obtain four extreme coordinate points, the leftmost, topmost, bottommost and rightmost coordinate points, by querying the pixels of the contour. The four extreme points are connected in sequence to obtain the extreme point straight line, and the extreme point straight line is converted into a linear extreme point curve through the Bezier curve algorithm. The total length of the extreme point curve is calculated by the combined weight based on the Euclidean distance to predict the length of the tobacco leaf.
[0051] like Figure 7As shown, preferably, step D1 comprises the following steps: E1. Introducing the zero-crossing point operator of the second-order derivative of the image intensity into the preset gradient formula, calculating the gradient value of each coordinate point, and extracting the tobacco leaf contour according to the gradient difference; E2. Perform pixel query on the tobacco leaf contour to obtain extreme points in different directions.
[0052] In step E1 to step E2, the zero-crossing point operator of the second-order derivative of the image intensity is introduced into the preset gradient formula, the gradient value of each coordinate point is calculated, and the tobacco leaf contour is extracted by the gradient difference of the coordinate points; the tobacco leaf contour is queried by pixels to obtain extreme points in different directions.
[0053] In this embodiment, it is necessary to first calculate the gradient difference of the pixel points of the grayscale image, and then obtain the contour points of the image, such as is the coordinate point of the image, and its function expression is , the gradient is the first-order partial derivative of the pixel, so the image The coordinate point is and The formula for calculating the gradient in the direction is:
[0054]
[0055] in The operator used in the present invention is the Marr-Hildreth operator, which is also called the zero-crossing point algorithm of the second-order derivative of image intensity. The advantage of this algorithm is that it is sensitive to noise and has a high extraction accuracy.
[0056] The core idea of the Marr-Hildreth operator is that the change in grayscale is independent of the image size, and that the change in grayscale of the image faces a peak in the first-order derivative or a zero crossing in the second-order derivative.
[0057] The standard deviation formula of the standard Gaussian function is:
[0058] right Second partial differential:
[0059]
[0060] Secondly, the cv2.minMaxLoc() function in OpenCV is used to query the four extreme points in the image, including the top, bottom, rightmost and leftmost points, and convert them into coordinate values using the numpy function.
[0061] In step D3, specifically: Due to the particularity of tobacco leaf morphology, the length of the extreme point straight line represents the actual length of the tobacco leaf, which will cause a large error and is not enough to represent the length of the tobacco leaf. Therefore, it is necessary to introduce and draw the Canvas Bezier curve algorithm. The concept of the Bezier curve is to draw a smooth curve through two points. This embodiment uses a quadratic Bezier curve, and its analytical diagram is as follows Figure 8 As shown: P0, P1, and P2 are called control points. The three points draw a smooth curve, and the beginning and end of the curve are connected to the starting point P0 and the end point P2 respectively. Q0 and Q1 are any points on the lines P0P1 and P1P2, so that the curve satisfies:
[0062] Then connect Q0 and Q1, where point B is any point on the straight line Q0Q1, and point B satisfies the formula:
[0063] The extreme point straight line is converted into an extreme point curve by using the quadratic Bezier curve algorithm.
[0064] Preferably, step D4 comprises the following steps: Define each round coordinate points, forming Line segments, the coordinate value cluster is ; Then the Euclidean distance formula for each line segment is: ; The total length of the curve is given by: .
[0065] Multiple curve segments are integrated according to the combined weight method based on Euclidean distance to obtain the total length of the curve.
[0066] In step D5, specifically, the actual size of the tobacco leaf can be calculated according to the ratio conversion between the pixel value of the image taken by the industrial camera and the actual length:
[0067]
[0068] Where A is the pixel value of the wide edge of the tobacco leaf captured by the industrial camera; a is the actual length captured by the industrial camera; β is the ratio between the pixel value and the actual length; and L is the actual length of the tobacco leaf.
[0069] Furthermore, the present invention also designs an inspection process, which is specifically: Artificial intelligence environment construction Install Anconda 3.0, use Python as the development language, and install OpenCV, Torch GPU version, Numpy, Labelme and other installation packages that can meet the model operation requirements, so that the model versions can be compatible with each other. Use Pycharm as the deep learning development IDE, and select the model environment in Pycharm as the virtual environment of Anconda.
[0070] Collecting data 3,000 high-definition tobacco leaf images taken by industrial cameras were collected, including 2,000 vertical tobacco leaves and 1,000 irregular tobacco leaves. Tobacco leaf grading hardware equipment and high-definition industrial cameras were used to collect high-definition image data of tobacco leaves.
[0071] Use the Labelme development kit in the Python environment to create the 1-class tobacco leaf segmentation data required by the Unet++ image semantic segmentation deep learning network architecture. Subtract the tobacco leaf part from the original tobacco leaf image along the edge, and finally save the corresponding Json package label for each image. The name of the Josn package is consistent with the name of the original image to facilitate the correspondence between data and labels. Prepare 3,000 data sets and labels.
[0072] Unet++ image semantic segmentation Build the Unet++ image semantic segmentation network architecture, divide 80% of the data set into a training set and 20% into a test set, where the training set is used for training and the test set is used for accuracy detection. Use Transform to convert the image into data of size (300, 300), and use Torch to convert the data into Tensor format, train 200 times to obtain the training weight with the highest accuracy, and edit the prediction model.
[0073] Tobacco leaf length prediction The Unet++ image semantic segmentation algorithm is used to extract the tobacco leaf part in the image, so that only the tobacco leaf part is retained in the image, and the other areas are filled with black. The image is grayed out using OpenCV, and the minimum rectangular bounding box is extracted; the minimum rectangular bounding box of 2000 vertical tobacco leaves is extracted and the average threshold is calculated to determine whether the tobacco leaf is in a vertical or irregular shape.
[0074] The four coordinate values in the numerical morphological image are (372, 434), (355, 42), (1781, -18), and (1798, 373), which are lower left, upper left, upper right, and lower right, respectively. One of the coordinate values is outside the image, and the use of negative coordinates does not affect the calculation of the length of the tobacco leaf. After calculating the ratio of the image attributes to the actual length, the length of the tobacco leaf is finally calculated to be 48.12 cm.
[0075] The Canvas Bezier curve method is closer to the actual length of the tobacco leaf. The four coordinate values in the irregular morphology image are (88, 26), (89, 25), (394, 63), and (229, 124). Finally, the length of each curve segment is calculated in turn by the combined weight method, and the predicted length of the tobacco leaf image is 37.56 cm. After measuring the actual length of the tobacco leaf, it is 39.05 cm. The absolute value of the difference between the two is divided by the actual length of the tobacco leaf to get the deviation rate. The calculated deviation rate is 0.03815, and the accuracy is 96.18%. The feasibility of this method is verified and the accuracy of the length prediction of irregular tobacco leaves is improved.
[0076] like Fig. 9 As shown, a tobacco leaf length prediction system comprises: A model building module is used to build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; An image segmentation module is used to obtain the original tobacco leaf image and input it into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; A tobacco leaf aspect ratio module is used to process the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; A tobacco leaf morphology module is used to determine the tobacco leaf morphology according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio; The tobacco leaf length module is used to calculate and obtain the tobacco leaf length corresponding to the tobacco leaf morphology.
[0077] In actual application, the present invention also discloses a tobacco leaf length prediction system. The functions of each module in the system correspond one-to-one to the steps of the tobacco leaf length prediction method, belong to the same technical field, solve the same technical problems, and should have the same beneficial effects, which will not be repeated here.
[0078] In the embodiments provided in the present application, it should be understood that the disclosed methods and systems can be implemented in other ways. The system embodiments described above are merely schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.
[0079] In addition, all functional modules in the embodiments of the present invention may be integrated into one processor, or each module may be a separate device, or two or more modules may be integrated into one device; each functional module in the embodiments of the present invention may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0080] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, the steps of the above method embodiment are executed; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a magnetic disk or an optical disk, and other media that can store program codes.
[0081] It should be understood that the use of "system", "device", "unit" and / or "module" in this application is only a method for distinguishing different components, elements, parts, parts or assemblies at different levels. However, if other words can achieve the same purpose, the word can be replaced by other expressions.
[0082] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements. The elements defined by the sentence "includes a..." do not exclude the existence of other identical elements in the process, method, commodity or device that includes the elements.
[0083] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.
[0084] If a flow chart is used in the present application, the flow chart is used to illustrate the operations performed by the system according to the embodiment of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, each step can be processed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or a certain step or several steps of operations can be removed from these processes.
[0085] The above is a detailed introduction to a tobacco leaf length prediction method and system provided by the present invention. The above description of the disclosed embodiments enables professionals and technicians in the field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in the field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features disclosed in the present invention.
Claims
1. A tobacco leaf length prediction method, characterized in that: The steps include: Build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; Obtaining an original tobacco leaf image, and inputting the image into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; Processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; Determining the shape of the tobacco leaves according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio; Calculate and obtain the tobacco leaf length corresponding to the tobacco leaf morphology.
2. The tobacco leaf length prediction method according to claim 1, characterized in that: The step of constructing and training the image semantic segmentation model to obtain the trained image semantic segmentation model includes the following steps: Construct an initial image semantic segmentation model consisting of convolutional layers, activation functions, and pooling layers, using sticky connections; The iterative training enables the initial image semantic segmentation model to converge, thereby obtaining a trained image semantic segmentation model.
3. The tobacco leaf length prediction method according to claim 1, characterized in that: The method of obtaining the original tobacco leaf image and inputting it into the trained image semantic segmentation model to obtain the segmented tobacco leaf image includes the following steps: Acquiring the original tobacco leaf image by an industrial camera installed at a fixed position; The trained image semantic segmentation model segments the input original tobacco leaf image to extract tobacco leaf regions as segmented tobacco leaf images.
4. The tobacco leaf length prediction method according to claim 3, characterized in that: The step of processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio comprises the following steps: The segmented tobacco leaf image is processed by grayscale and binarization in sequence by the image processing tool to obtain a tobacco leaf image matrix; Extracting tobacco leaf contours according to the image processing tool and the tobacco leaf image matrix; According to the tobacco leaf contour and the minimum circumscribed rectangular frame method, the endpoint coordinate values of the minimum circumscribed rectangular frame are obtained; Repeat the above steps to obtain the actual value of the tobacco leaf aspect ratio according to the endpoint coordinate values of the multiple minimum circumscribed rectangular frames and the preset formula.
5. The tobacco leaf length prediction method according to claim 4, characterized in that: The step of determining the tobacco leaf morphology according to the actual value of the tobacco leaf length-width ratio and the preset threshold value of the tobacco leaf length-width ratio comprises the following steps: Determining whether the actual value of the tobacco leaf length-to-width ratio is not greater than the tobacco leaf length-to-width ratio threshold; If yes, the tobacco leaf is defined to be in a vertical position; If not, the tobacco leaf is defined as being in an irregular shape.
6. The tobacco leaf length prediction method according to claim 5, characterized in that: The step of calculating and obtaining the tobacco leaf length corresponding to the tobacco leaf morphology comprises the following steps: When the tobacco leaf is in a vertical state, the first tobacco leaf length is calculated and obtained according to the coordinate values of the endpoints of the minimum circumscribed rectangular frame.
7. The tobacco leaf length prediction method according to claim 5, characterized in that: The step of calculating and obtaining the tobacco leaf length corresponding to the tobacco leaf shape further includes the following steps: When the tobacco leaf is in an irregular shape, extracting extreme points in different directions from the contour of the tobacco leaf; Connecting the extreme points in different directions of the tobacco leaf contour in sequence to obtain an extreme point straight line; According to the Bezier curve algorithm, the extreme point straight line is converted into an extreme point curve; Calculating the total length of the extreme point curve according to a combined weight method based on Euclidean distance; The second tobacco leaf length is obtained according to the parameters of the industrial camera and the total length of the extreme point curve.
8. The tobacco leaf length prediction method according to claim 7, characterized in that: The step of extracting extreme points in different directions from the tobacco leaf contour comprises the following steps: The zero-crossing point operator of the second-order derivative of the image intensity is introduced into the preset gradient formula to calculate the gradient value of each coordinate point, and the contour of the tobacco leaf is extracted according to the gradient difference; A pixel query is performed on the tobacco leaf contour to obtain extreme points in different directions.
9. The tobacco leaf length prediction method according to claim 7, characterized in that: The method of calculating the total length of the extreme point curve according to the combined weight method based on Euclidean distance comprises the following steps: Define each round coordinate points, forming Line segments, the coordinate value cluster is ; Then the Euclidean distance formula for each line segment is: ； The total length of the curve is given by: 。 10. A tobacco leaf length prediction system, characterized in that: include: A model building module is used to build and train an image semantic segmentation model to obtain a trained image semantic segmentation model; An image segmentation module is used to obtain an original tobacco leaf image and input it into the trained image semantic segmentation model to obtain a segmented tobacco leaf image; A tobacco leaf aspect ratio module, used for processing the segmented tobacco leaf image according to an image processing tool to obtain an actual value of the tobacco leaf aspect ratio; A tobacco leaf morphology module, used to determine the tobacco leaf morphology according to the actual value of the tobacco leaf length-width ratio and a preset threshold value of the tobacco leaf length-width ratio; The tobacco leaf length module is used to calculate and obtain the tobacco leaf length corresponding to the tobacco leaf morphology.