A rice seed recognition and counting method based on image detection

Abstract

The invention provides a rice seed recognition and counting method based on image detection, wherein the method adopts grid paper as the background for rice seed image collection, thereby realizing image positioning and correction; then, color components with significant differences between rice seeds and background are selected to form SbCbCr color space, and the rice seed image is segmented into binary images in combination with a BPNN classification model; after opening and closing operations, the original image is divided into sub-images of a single rice seed or a plurality of adhered rice seeds using a connected region of the binary image as a mask, and the size is normalized; a rice seed quantity image data set is established from the sub-images, and the collected rice seed images are classified using a deep convolutional neural network classification model, and then the number of rice seeds corresponding to the classification result of each sub-image in the original rice seed image is counted to obtain the total number of rice seeds in the original rice seed image. The invention is simple to operate, has accurate counting, can save labor costs and equipment costs, and greatly improves work efficiency.

Description

Technical Field

[0001] The invention belongs to the field of image processing, and in particular relates to a rice seed recognition and counting method based on image detection. Background Art

[0002] Accurately counting the number of seeds is an indispensable step in modern agricultural research and an important part of breeding and testing. Crop seed research mainly focuses on quality detection and counting. Seed identification and segmentation are the most important links in crop seed counting. Most studies focus on segmenting adherent crop seeds, among which segmentation algorithms based on morphology, concave point detection, and elliptic curve fitting are basic methods. There are also other algorithms, such as segmentation algorithms based on seed region growth, segmentation algorithms based on clustering algorithms and watershed algorithms. However, the current counting method has high requirements on the environment and images, and the speed is relatively slow, so its convenience, accuracy, and speed need to be improved.

[0003] At present, the manual counting method is still very common, and its advantages are high real-time performance and low tool requirements, but its disadvantages are low efficiency, easy to make mistakes, large labor force, and easy to cause visual fatigue. There are also a few mechatronic grain counting devices that can replace manual counting, but there are problems such as large errors, complex manufacturing, and high prices, which are difficult to be widely promoted and applied. In summary, in order to solve the deficiencies of the above-mentioned prior art, the present invention provides a rice seed recognition and counting method based on image detection on the basis of image processing technology. Summary of the invention

[0004] Purpose of the invention: In view of the shortcomings of the prior art, the present invention proposes a rice seed recognition and counting method based on image detection, which uses a deep convolutional neural network classification model to realize the counting of rice seed adhesion, achieving the effects of fast speed, high accuracy and strong stability.

[0005] Technical solution: The present invention provides a rice seed recognition and counting method based on image detection, which specifically comprises the following steps:

[0006] (1) The rice seeds are evenly scattered on white grid paper, the camera lens is parallel to the top of the grid paper to shoot from above, and a parallel light source is used for fill light;

[0007] (2) Rotate and crop the image according to the positions of the four corner positioning blocks of the grid paper, and then calibrate the image distortion according to the positions of the grid intersections in the grid paper;

[0008] (3) Extract the S component of the rice seed image in the HSV color space, the b component of the Lab color space, and the Cb and Cr components of the YCbCr color space and combine them into a new color space SbCbCr;

[0009] (4) Build a BPNN pixel classification model, then extract the pixel values ​​of the seed pixels and background pixels in the sample image in four channels in the newly created color space SbCbCr, train the BPNN model, and select the optimal model;

[0010] (5) The color values ​​of the four channels of each pixel in the rice seed image in the SbCbCr color space are used as the input of the optimal BPNN model to segment the rice seed image into a binary image, in which the rice seed is segmented as the foreground and the grid paper is segmented as the background;

[0011] (6) Opening operation is used to disconnect the small adhesions of some rice seed connected areas and filter out tiny noise points, and then closing operation is used to make the rice seed edges smoother;

[0012] (7) using each rice seed connected region as a mask, dividing the original rice seed image into rice seed images consisting of a single rice seed or a plurality of connected rice seeds, and then normalizing the size of the images;

[0013] (8) Label the number of rice seeds in each sub-image to establish a rice seed quantity image dataset, use a deep convolutional neural network to establish an image classification model, and input the image dataset into the classification model for training;

[0014] (9) The collected rice seed image is divided into multiple sub-images and sent to the trained classification model. Then, the number of rice seeds corresponding to the classification result of each rice seed image in the original rice seed image is counted to obtain the total number of rice seeds in the original rice seed image.

[0015] Furthermore, the implementation process of step (2) is as follows:

[0016] (21) Segmenting the positioning block from the image according to the color of the positioning block on the grid paper;

[0017] (22) Using the corner point detection method to determine the vertex position of each positioning block;

[0018] (23) Connect the vertices of the four positioning blocks and calculate the deflection angle of the grid paper;

[0019] (24) performing reverse rotation and cropping operations on the collected image according to the deflection angle, thereby straightening the image;

[0020] (25) Corner point detection is used to determine the position of each intersection in the grid. The horizontal and vertical distances between adjacent intersections at different positions in the image are calculated and subtracted from the actual side length of the grid in the image to calculate the horizontal and vertical distortion rate of the image.

[0021] (26) Image calibration is performed based on the distortion rate.

[0022] Furthermore, the implementation process of step (3) is as follows:

[0023] The calculation methods of different color components, the calculation formula of the S component in the HSV color space is shown in formula (1), the calculation formula of the b component in the Lab color space is shown in formula (2), and the calculation formulas of the Cb and Cr components in the YCbCr color space are shown in formula (4) respectively;

[0024]

[0025] b=200(h(Y / Y w )-h(Z / Z w )) (2)

[0026]

[0027] Where MAX and MIN are the maximum and minimum values ​​of the three color components in the RGB color space, respectively; Y and Z are the corresponding components in the XYZ color space, respectively. w and Z w The reference values ​​are 1.0000 and 1.0888 respectively, where the color stimulus value calibration function h(t) is shown in formula (3); R, G, and B are the three components of the RGB color space respectively.

[0028] Furthermore, the implementation process of step (4) is as follows:

[0029] (41) The number of nodes in the input layer of the BPNN model is 4, and the four color channel components of each pixel in the SbCbCr color space are used as input data;

[0030] (42) The number of nodes in the output layer of the BPNN model is 1, and the rice seed pixel samples are marked as 1 as positive samples, and the background pixel samples are marked as 0 as negative samples;

[0031] (43) The number of hidden layer nodes of the BPNN model was set to 10, the activation function of the hidden layer was set to Sigmoid, the activation function of the output layer was set to Softmax, and the cross entropy loss function was used to measure the prediction error;

[0032] (44) The quantized conjugate gradient function was used for multiple trainings, and the loss, error, accuracy, true positive rate, and false positive rate of different training models were calculated and analyzed to select the optimal BPNN model.

[0033] Furthermore, the implementation process of step (5) is as follows:

[0034] (51) Convert the M*N rice seed image into a (M*N, 4) matrix, where each row of the matrix corresponds to a pixel of the image, and each column of the matrix corresponds to a color component in the SbCbCr color space;

[0035] (52) Normalize the matrix data and input it into the optimal BPNN model;

[0036] (53) Quantize the M*N dimensional column vector output by the model, set the value greater than or equal to 0.5 to 1, and set the value less than 0.5 to 0;

[0037] (54) The column vector is converted into a binary image of size M*N, where the pixel points with a value of 1 are foreground and represent rice seeds, and the pixel points with a value of 0 are background and represent grid paper.

[0038] Furthermore, the implementation process of step (7) is as follows:

[0039] (71) Calculate the coordinates of the topmost, bottommost, leftmost and rightmost pixels of each connected region;

[0040] (72) Draw an outer rectangle according to the pixel coordinates in four directions;

[0041] (73) mapping the bounding rectangle in the binary image to the original image, and cutting out a sub-image containing a single rice seed or a plurality of adhered rice seeds from a corresponding area in the original image;

[0042] (74) Normalize the size of the cropped sub-image.

[0043] Beneficial effects: Compared with the prior art, the present invention has the following beneficial effects: the present invention applies digital image processing technology to seed counting in the breeding field, replacing manual counting and mechanical counting methods, saving labor costs and equipment costs, and greatly improving work efficiency; the present invention uses grid paper with positioning blocks to accurately identify and segment rice seeds; the present invention uses a deep convolutional neural network classification model to realize the counting of seed adhesion, achieving the advantages of fast speed, high accuracy, strong stability, batch processing, etc.; the present invention is easy to operate, low in cost, and can achieve accurate prediction of the number of rice seeds. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flow chart of the present invention;

[0045] Figure 2 It is a flowchart for establishing a BPNN pixel classification model;

[0046] Figure 3 is the rice seed binary image;

[0047] Figure 4 It is a rice seed morphological processing image. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0049] The invention proposes a rice seed recognition and counting method based on image detection, wherein the rice seeds are evenly scattered on white grid paper, a camera lens is parallel to the grid paper for overhead shooting, and a parallel light source is used for fill light. The collected image is imported into a computer, the image is rotated and cropped according to the positions of the four corner positioning blocks of the grid paper, and then the image distortion is calibrated according to the positions of the grid intersections in the grid paper; the S component of the rice seed image in the HSV color space, the b component of the Lab color space, and the Cb and Cr components of the YCbCr color space are extracted to form a new color space SbCbCr. A BPNN pixel classification model is established, and then the pixel values ​​of the seed pixels and background pixels in the sample image in four channels in the newly created color space SbCbCr are extracted, the BPNN model is parameter trained, and the optimal model is selected; the color values ​​of the four channels of each pixel in the rice seed image in the SbCbCr color space are used as the input of the optimal BPNN model, and the rice seed image is segmented into a binary image, wherein the rice seeds are segmented as the foreground and the grid paper is segmented as the background. An opening operation is used to disconnect the small adhesions of some rice seed connected areas and filter out tiny noise points, and then a closing operation is used to make the edges of the rice seeds smoother. Using each rice seed connected area as a mask, the original rice seed image is divided into rice seed images consisting of a single rice seed or multiple adhered rice seeds, and then the size is normalized; the rice seeds in each sub-image are labeled to establish a rice seed quantity image dataset, and a deep convolutional neural network is used to establish an image classification model, and the image dataset is input into the classification model for training. The collected rice seed image is divided into multiple sub-images and sent to the trained classification model, and then the number of rice seeds corresponding to the classification result of each rice seed image in the original rice seed image is counted to obtain the total number in the original rice seed image. Figure 1 As shown, the specific steps include:

[0050] Step 1: Sprinkle the rice seeds evenly on the white grid paper, shoot from above with the camera lens parallel to the grid paper, and use parallel light source for fill light to make the collected image clear and bright and reduce the influence of shadows. Use grid paper with positioning blocks as the background for shooting, and use parallel light source to reduce the generation of shadows.

[0051] Step 2: Rotate and crop the image according to the positions of the four corner positioning blocks of the grid paper, and then calibrate the image distortion according to the positions of the grid intersections in the grid paper.

[0052] According to the color of the grid paper positioning block, the positioning block is separated from the image; the corner detection method is used to determine the vertex position of each positioning block; the vertices of the four positioning blocks are connected, and the deflection angle of the grid paper is calculated; according to the deflection angle, the collected image is reversely rotated and cropped to straighten the image. Corner detection is used to determine the position of each intersection in the grid, and then the distance between adjacent corner points in the horizontal and vertical directions is calculated respectively, and it is subtracted from the actual side length of the grid in the image to calculate the distortion rate of the image in the horizontal and vertical directions, and finally the image is calibrated according to the distortion rate.

[0053] Step 3: Extract the S component of the rice seed image in the HSV color space, the b component of the Lab color space, and the Cb and Cr components of the YCbCr color space and combine them into a new color space SbCbCr.

[0054] The calculation methods of different color components, the calculation formula of the S component in the HSV color space is shown in formula (1), the calculation formula of the b component in the Lab color space is shown in formula (2), and the calculation formulas of the Cb and Cr components in the YCbCr color space are shown in formula (4) respectively.

[0055]

[0056] b=200(h(Y / Y w )-h(Z / Z w )) (2)

[0057]

[0058] Where MAX and MIN are the maximum and minimum values ​​of the three color components in the RGB color space, respectively; Y and Z are the corresponding components in the XYZ color space, respectively. w and Z w The reference values ​​are 1.0000 and 1.0888 respectively, where the function color stimulus value calibration function h(t) is shown in Formula 3; R, G, and B are the three components of the RGB color space respectively.

[0059] Step 4: Establish a BPNN pixel classification model, then extract the pixel values ​​of the four channels of the seed pixels and background pixels in the sample image in the newly created color space SbCbCr, train the parameters of the BPNN model, and select the optimal model.

[0060] Training the BPNN pixel classification model is as follows Figure 2 As shown, including:

[0061] S1: The number of nodes in the input layer of the BPNN model is 4, and the four color channel components of each pixel in the SbCbCr color space are used as input data.

[0062] S2: The number of nodes in the output layer of the BPNN model is 1. The rice seed pixel samples are marked as 1 as positive samples, and the background pixel samples are marked as 0 as negative samples.

[0063] S3: The number of hidden layer nodes of the BPNN model is set to 10, the activation function of the hidden layer is set to Sigmoid, the activation function of the output layer is set to Softmax, and the cross entropy loss function is used to measure the prediction error.

[0064] S4: Use the quantized conjugate gradient function to perform multiple trainings, and calculate and analyze the loss, error, accuracy, true positive rate, and false positive rate of different training models to select the optimal BPNN model.

[0065] Step 5: Use the color values ​​of the four channels of each pixel in the rice seed image in the SbCbCr color space as the input of the optimal BPNN model to segment the rice seed image into a binary image, such as Figure 3 As shown, the rice seeds are segmented as the foreground and the grid paper is segmented as the background.

[0066] Convert the M*N rice seed image into a (M*N, 4) matrix, where each row of the matrix corresponds to a pixel in the image, and each column of the matrix corresponds to a color component in the SbCbCr color space. Normalize the matrix data and input it into the optimal BPNN model. Quantize the M*N dimensional column vector output by the model, set values ​​greater than or equal to 0.5 to 1, and values ​​less than 0.5 to 0. Convert the column vector into a binary image of size M*N. Figure 3 The pixel points with a value of 1 in the figure are foreground, representing rice seeds, and the pixel points with a value of 0 in the figure are background, representing grid paper.

[0067] Step 6: Use the opening operation to disconnect the small adhesions of the connected areas of the rice seeds and filter out the tiny noise points, and then use the closing operation to make the edges of the rice seeds smoother. Select a square structure element with a side length of 2 for the opening operation, and select a square structure element with a side length of 2 for the closing operation. The image after the opening and closing operations is as follows: Figure 4 shown.

[0068] Step 7: Using each rice seed connected region as a mask, the original rice seed image is divided into rice seed images consisting of a single rice seed or multiple adhered rice seeds, and then the images are size normalized.

[0069] Calculate the coordinates of the top, bottom, left and right pixels of each connected region; draw the bounding rectangle according to the pixel coordinates in four directions. Map the bounding rectangle in the binary image to the original image, and cut out a sub-image containing a single rice seed or multiple connected rice seeds in the corresponding area of ​​the original image; normalize the size of the cut sub-image.

[0070] Step 8: Label the number of rice seeds in each sub-image to establish a rice seed quantity image dataset, use a deep convolutional neural network to build an image classification model, and input the image dataset into the classification model training.

[0071] Label the number of rice seeds in each sub-image and establish a rice seed number image dataset; establish a deep convolutional neural network classification model, the input of the model is consistent with the image size of the dataset, and the output is the number of rice seeds; send the dataset images to the deep convolutional neural network model for multiple training, test the convolutional neural network model, and obtain a trained convolutional neural network model.

[0072] Step 9: Divide the collected rice seed image into multiple sub-images and send them into the trained classification model, then count the number of rice seeds corresponding to the classification result of each rice seed image in the original rice seed image to obtain the total number of rice seeds in the original rice seed image.

[0073] The rice seed images are collected and divided into size-normalized sub-images containing single or multiple adhered rice seeds; the normalized sub-images are sent to the trained image classification model for classification; the classification results of each sub-image are counted and summed to obtain the accurate number of rice seeds in the collected image.

[0074] The method has accurate calculation and simple operation, can effectively count rice seeds accurately, saves labor costs and equipment costs, and greatly improves work efficiency.

[0075] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A rice seed recognition and counting method based on image detection, characterized in that: The following steps are involved: (1) The rice seeds are evenly scattered on white grid paper, the camera lens is parallel to the top of the grid paper to shoot from above, and a parallel light source is used for fill light; (2) Rotate and crop the image according to the positions of the four corner positioning blocks of the grid paper, and then calibrate the image distortion according to the positions of the grid intersections in the grid paper; (3) Extract the S component of the rice seed image in the HSV color space, the b component of the Lab color space, and the Cb and Cr components of the YCbCr color space and combine them into a new color space SbCbCr; (4) Build a BPNN pixel classification model, then extract the pixel values ​​of the seed pixels and background pixels in the sample image in four channels in the newly created color space SbCbCr, train the BPNN model, and select the optimal model; (5) The color values ​​of the four channels of each pixel in the rice seed image in the SbCbCr color space are used as the input of the optimal BPNN model to segment the rice seed image into a binary image, in which the rice seed is segmented as the foreground and the grid paper is segmented as the background; (6) Opening operation is used to disconnect the small adhesions of some rice seed connected areas and filter out tiny noise points, and then closing operation is used to make the rice seed edges smoother; (7) using each rice seed connected region as a mask, dividing the original rice seed image into rice seed images consisting of a single rice seed or a plurality of connected rice seeds, and then normalizing the size of the images; (8) Label the number of rice seeds in each sub-image to establish a rice seed quantity image dataset, use a deep convolutional neural network to establish an image classification model, and input the image dataset into the classification model for training; (9) The collected rice seed image is divided into a plurality of sub-images and sent to a trained classification model, and then the number of rice seeds corresponding to the classification result of each rice seed image in the original rice seed image is counted to obtain the total number of rice seeds in the original rice seed image.

2. The rice seed recognition and counting method based on image detection according to claim 1, characterized in that: The implementation process of step (2) is as follows: (21) Segmenting the positioning block from the image according to the color of the positioning block on the grid paper; (22) Using the corner point detection method to determine the vertex position of each positioning block; (23) Connect the vertices of the four positioning blocks and calculate the deflection angle of the grid paper; (24) performing reverse rotation and cropping operations on the collected image according to the deflection angle, thereby straightening the image; (25) Corner point detection is used to determine the position of each intersection in the grid. The horizontal and vertical distances between adjacent intersections at different positions in the image are calculated and subtracted from the actual side length of the grid in the image to calculate the horizontal and vertical distortion rate of the image. (26) Image calibration is performed based on the distortion rate.

3. The rice seed recognition and counting method based on image detection according to claim 1, characterized in that: The implementation process of step (3) is as follows: The calculation methods of different color components, the calculation formula of the S component in the HSV color space is shown in formula (1), the calculation formula of the b component in the Lab color space is shown in formula (2), and the calculation formulas of the Cb and Cr components in the YCbCr color space are shown in formula (4) respectively; b=200(h(Y / Y w )-h(Z / Z w )) (2) Where MAX and MIN are the maximum and minimum values ​​of the three color components in the RGB color space, respectively; Y and Z are the corresponding components in the XYZ color space, respectively. w and Z w The reference values ​​are 1.0000 and 1.0888 respectively, where the color stimulus value calibration function h(t) is shown in formula (3); R, G, and B are the three components of the RGB color space respectively.

4. The rice seed recognition and counting method based on image detection according to claim 1, characterized in that: The implementation process of step (4) is as follows: (41) The number of nodes in the input layer of the BPNN model is 4, and the four color channel components of each pixel in the SbCbCr color space are used as input data; (42) The number of nodes in the output layer of the BPNN model is 1, and the rice seed pixel samples are marked as 1 as positive samples, and the background pixel samples are marked as 0 as negative samples; (43) The number of hidden layer nodes of the BPNN model was set to 10, the activation function of the hidden layer was set to Sigmoid, the activation function of the output layer was set to Softmax, and the cross entropy loss function was used to measure the prediction error; (44) The quantized conjugate gradient function was used for multiple trainings, and the loss, error, accuracy, true positive rate, and false positive rate of different training models were calculated and analyzed to select the optimal BPNN model.

5. The rice seed recognition and counting method based on image detection according to claim 1, characterized in that: The implementation process of step (5) is as follows: (51) Convert the M*N rice seed image into a (M*N, 4) matrix, where each row of the matrix corresponds to a pixel of the image, and each column of the matrix corresponds to a color component in the SbCbCr color space; (52) Normalize the matrix data and input it into the optimal BPNN model; (53) Quantize the M*N dimensional column vector output by the model, set the value greater than or equal to 0.5 to 1, and set the value less than 0.5 to 0; (54) The column vector is converted into a binary image of size M*N, where the pixel points with a value of 1 are foreground and represent rice seeds, and the pixel points with a value of 0 are background and represent grid paper.

6. The rice seed recognition and counting method based on image detection according to claim 1, characterized in that: The implementation process of step (7) is as follows: (71) Calculate the coordinates of the topmost, bottommost, leftmost and rightmost pixels of each connected region; (72) Draw an outer rectangle according to the pixel coordinates in four directions; (73) mapping the bounding rectangle in the binary image to the original image, and cutting out a sub-image containing a single rice seed or a plurality of adhered rice seeds from a corresponding area in the original image; (74) Normalize the size of the cropped sub-image.

Images