A device and method for monitoring seedling quality grading based on image and near-infrared spectral phenotypes

Abstract

This invention discloses a seedling quality grading monitoring device and method based on image and near-infrared spectral phenotypes. The monitoring device includes: a truss, a camera module, a near-infrared spectral sensor module, a development board, a classification action device, a step-down board, a battery, and an API server. The monitoring method includes: acquiring RGB images of seedlings through the camera; obtaining quantitative data such as leaf size, vein length, and yellow leaf spot area through image processing; acquiring spectral curves through the near-infrared spectral sensor module; extracting quantitative data through signal processing; and then processing and classifying the data, loading it into a classification model for classification, and determining the seedling quality. This invention achieves black-box interface access and hostless operation, reducing device size and cost while improving algorithm speed. Through mobile APP development and web-based host computer development, the hardware device is mountable while reducing learning costs.

Description

Technical Field

[0001] This invention relates to a seedling quality grading and monitoring device and method based on image and near-infrared spectral phenotypes, belonging to the field of smart agriculture technology. Background Technology

[0002] Smart agriculture mainly includes precision agriculture in field fields, smart animal husbandry, smart fisheries, and intelligent greenhouses. Key research areas in precision agriculture include remote sensing and sensor technology, agricultural big data and cloud computing services, and intelligent agricultural equipment. Phenotypic detection is an upstream technology for intelligent agricultural equipment and systems, serving as a precursor to modern agricultural machinery such as fertigation. Seedling quality phenotypes are reflected in leaf characteristics such as size, shape, veins, color, and lesions. Research covers a wide range of aspects; therefore, quantifying seedling leaf phenotypes is a crucial part of realizing smart agricultural machinery. Previous agricultural machinery research has focused less on seedling phenotypic detection, with some detection still requiring manual labor. Furthermore, the application of computer vision and near-infrared phenotypic detection for seedlings requires significant technical expertise for automated detection. Moreover, without the support of IoT technology, collaboration with other agricultural machinery is difficult.

[0003] Currently, there has been some research and development in seedling type detection in China. Li Zimao et al. improved the YOLOv3 algorithm model to adapt to multi-leaf detection of roses in complex backgrounds, and then used K-means to optimize Faster R-CNN for rose pest and disease detection. Experimental results show that the optimized network model can achieve an average detection accuracy of 82.26%. However, current visual technology has advanced to YOLOv5, while the recognition rate of the research results is too low, and phenotypic extraction and quantification are not performed. Chinese patent 2019104375625 discloses a plant weed identification system using computer vision, which identifies and labels weeds. The core technology can be used for seedling image segmentation, but there is no further phenotypic quantification algorithm or machine learning evaluation. Chinese patent 2012104987672 discloses a method for identifying, locating, and transplanting plug seedlings. This method uses a transplanter to determine the seedling vigor, but leaf area measurement requires manual measurement, which is difficult to operate and has low measurement accuracy. None of the above studies have achieved integration with the Internet of Things, and they have also failed to comprehensively meet the needs of seedling leaf phenotyping from acquisition to quantification to machine learning classification. Summary of the Invention

[0004] The purpose of this invention is to provide a seedling quality grading and monitoring device and method based on image and near-infrared spectral phenotypes, which effectively improves the speed and accuracy of phenotypic detection and reduces the manpower consumption of phenotypic detection.

[0005] The technical solution adopted in this invention is as follows:

[0006] On one hand, this invention provides a seedling quality grading and monitoring device based on computer vision and near-infrared spectral phenotypes, characterized in that it includes a truss, a camera module, a near-infrared spectral sensor module, a development board, a step-down board, a battery, and an API server; wherein:

[0007] The truss serves as the hardware placement structure for the device;

[0008] The camera module is fixed directly below the truss beam and is used to take pictures of rice seedlings and upload the image data to the server via the network.

[0009] The near-infrared spectral sensor module is located inside the truss support arm and is used to obtain the near-infrared spectrum of the light reflected from the seedling leaf surface from the side of the seedling.

[0010] The development board serves as the control center, used for communication, sending commands, receiving data from modules, and transmitting data to the server.

[0011] The step-down plate is used to provide the device with a suitable operating voltage;

[0012] The battery is used to power the device when it is operating outdoors;

[0013] The API server is an interface server located in the cloud, used to exchange data and run program algorithms through interface access.

[0014] Furthermore, the monitoring device also includes a classification action device, which serves as a seedling quality screening drive mechanism for grading and then eliminating seedlings.

[0015] On the other hand, the present invention also provides a method for monitoring seedling quality grading based on computer vision and near-infrared spectral phenotypes, characterized by the following steps:

[0016] Step 1: Train the recognition model using the YOLOv5s training method; train the classification model using SVM machine learning.

[0017] Step 2: After placing the seedlings on the testing platform, press the "Take Photo" button on the host computer to send a command to acquire the image;

[0018] Step 3: After taking a picture, access the server's image upload interface and upload the picture. Then the picture is saved to the server and image distortion correction is performed.

[0019] Step 4: The server calls the YOLOv5 recognition model to analyze the image and check whether there are actually seedlings in the image to prevent malicious or incorrect use. If there are no seedlings, an err message will be output to the device, requesting the user to retake the image; otherwise, proceed to the next step.

[0020] Step 5: The server sends a command to use the near-infrared spectral sensor module to acquire the near-infrared spectral curve of the seedling from the leaf end, and performs frequency domain transformation on the near-infrared spectrometer. The curve is then converted into a frequency domain range sequence using the Fast Fourier Transform (FFT) method, and then the server saves and receives the data.

[0021] Step 6: The server calls the phenotypic quantization processing command interface to start the image processing algorithm to quantize the image phenotypic quantization.

[0022] Step 7: The server calls the phenotypic quantization processing command interface to start the signal processing algorithm to quantize the near-infrared spectral phenotypic quantization.

[0023] Step 8: The server calls the classification command interface to process and classify all the data, loads it into the classification model after SVM machine learning, and obtains the quality status of the seedlings.

[0024] Step 9: The server calls the driver command interface and sends the corresponding driver commands for the three levels, A, B and C, to the device according to the quality results, so that the device can operate the classification driver to perform classification actions.

[0025] Step 10: The server calls the tabular output interface to send the results back to the host computer, which then obtains the classification and detection results.

[0026] Furthermore, the training process of the recognition model is as follows:

[0027] Step 101: Randomly select a certain number of seedlings and screen them, dividing them into three different quality groups: A, B, and C;

[0028] Step 102: Take RGB images of the seedlings from the top of each group to obtain a seedling image sample set;

[0029] Step 103: Select the main body of the seedlings in all the seedling images, mark them with a uniform label, and generate a text file containing the coordinates of the four corners of the selection box and the label;

[0030] Step 104: Match the images and label text files one-to-one as the dataset, and divide it into two parts, training set and test set, according to the ratio;

[0031] Step 105: Set up the training program, take the dataset as input, and select the yolov5s training method to train the model; after training, check the training result report until the actual seedling recognition rate reaches more than 90%.

[0032] Furthermore, the learning and training process of the classification model is as follows:

[0033] Step 106: Perform image preprocessing on the seedling image sample set obtained in Step 102;

[0034] Step 107: Enhance the color of the sample set;

[0035] Step 108: Extract phenotypic masks from the sample set;

[0036] Step 109: Perform phenotypic mask edge detection on the sample set;

[0037] Step 110: Quantize the phenotypic images of the sample set;

[0038] Step 111: Obtain the near-infrared spectrum of the seedling leaves;

[0039] Step 112: Perform first-order guide spectrum calculation, calculation formula: Where: L(w) is the near-infrared spectral curve, L′(w) is the first derivative of the spectral curve, w0 is the x-coordinate of the sampling point, and the first derivative of the sampling point is used as the spectral quantization value.

[0040] Step 113: Take the quantification results and classification of each plant sample as samples, make a dataset, and divide it into training set and test set according to the proportion.

[0041] Step 114: Import the dataset using the program and perform SVM machine learning training. After learning, obtain the trained model and model accuracy until the accuracy reaches more than 95%, which meets the requirements.

[0042] Step 115: Save the model files obtained after training YOLOv5 and SVM and load them for use.

[0043] Furthermore, the image preprocessing uses Gaussian blur to remove noise from the image, using the formula: g(i,j)=h(i,j)*f(i,j). Arranging g(i,j) in the order of the images to form an image matrix yields the noise-removed image, where i and j are the pixel sizes, h(i,j) is a Gaussian smoothing filter set to a 3×3 matrix, f(i,j) are the individual pixels of the original image, and g(i,j) are the individual pixels of the convolved image.

[0044] Furthermore, the color enhancement uses the formula: fin=αG+βB+γR to enhance the colors of the phenotype to be extracted, where fin is the enhanced image, α, β, and γ are coefficients, and G, B, and R are the green, blue, and red monochrome channel images, respectively. The phenotypic parameters for extracting the plant body are set to α=1, β=-1, and γ=0.6, and the phenotypic parameters for extracting the withered parts are set to α=-1, β=0, and γ=1.

[0045] Furthermore, the phenotypic mask extraction uses Otsu's method to obtain the optimal threshold, converting the phenotypic enhanced image into a binary image, and then uses opening and closing operations to fit the edges of the image. Finally, color inversion processing is performed to obtain the mask for the entire seedling and yellow leaves. The phenotypic mask edge detection involves applying the Sobel edge detection operator to all masks to extract the edge coordinate matrix of the mask. The function is implemented using the following formula: Among them, G x and G y The image grayscale values ​​after horizontal and vertical edge detection are given, where A is the original image matrix, x and y are the horizontal and vertical coordinates of specific pixels in the image, G is the gradient, and θ is the gradient direction.

[0046] Furthermore, the phenotypic image is quantized, and the number of withered yellow spots is obtained by counting the number of masks in the withered yellow areas; the pixel area of ​​the mask is calculated by accumulating the pixels in the closed edge, and then the actual phenotypic area is calculated. The function is implemented with the following formula: Where g is the actual area of ​​the mask, f is the number of pixels in the mask, and w 2 r is the pixel area, usually set to 1. 2 To map a pixel to its actual area in reality, calculate the leaf area of ​​the entire seedling and the area of ​​each withered yellow spot, and then calculate their ratio, which is the yellow leaf ratio.

[0047] Furthermore, the image distortion correction method is as follows: The intrinsic and extrinsic parameters of the camera are obtained by measuring the coordinate points using the Zhang Zhengyou calibration method, and then calculated using the formula: r 2 =x 2 +y 2 Radial distortion correction formula: x dr =x(1+k1r) 2 +k2r 4 +k3r 6 ), y dr =y(1+k1r 2 +k2r 4 +k3r 6 ), Tangential distortion correction formula: x dt =2p1xy+p2(r 2 +2x 2 )+1, y dt =2p1(r 2 +2x 2 The algorithm )+2p2xy+1 yields a fixed computation matrix, which is used to calculate the corrected image from the input image; where r is the imaging center distance, x dr y dr x dt ydt Let x and y be the coordinates of the image to be corrected, k1, k2, k3, p1, and p2 be the coefficients of the distortion response, and x and y be the corrected coordinates.

[0048] The beneficial effects of this invention are as follows:

[0049] (1) This invention combines multiple technologies such as computer image, near-infrared spectroscopy, and machine learning to effectively improve the speed and accuracy of phenotypic detection quantification and reduce the manpower consumption of phenotypic detection.

[0050] (2) Based on machine learning technology, this invention realizes the quality judgment and grading of seedlings, and takes into account multiple usage scenarios. It can be customized for different types of finished seedlings and has strong compatibility.

[0051] (3) The present invention transfers the algorithm operation part to the cloud, which reduces the hardware requirements of the device for operation, thereby reducing costs and making it more economical;

[0052] (4) This invention implements Internet of Things (IoT) functions based on API servers. It can coordinate multiple machines and collaborate with other production equipment through interfaces. It is compatible with all host computers that can be connected to the Internet, thus effectively realizing the goal of smart agriculture. Attached Figure Description

[0053] Figure 1 and Figure 2 These are schematic diagrams of the monitoring device of the present invention from different perspectives;

[0054] Figure 3 This is a flowchart of the classification method of the present invention;

[0055] Figure 4 This is a flowchart of the model training process of the present invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0057] Taking tomato seedling quality classification as an example, a factory setting is used, with an Android APP serving as the host computer for manual operation. Customization is performed based on the seedling condition, determining that leaf area, number of withered yellow spots, yellow leaf ratio, and spectral quantification value are used as feature values ​​for classification. The classification results are divided into three levels: A, B, and C.

[0058] like Figure 1 As shown, the seedling quality grading and monitoring device based on image and near-infrared spectral phenotype includes: truss 1, camera module 2, near-infrared spectral sensor module 3, EPS32 chip development board 4, classification action device 5, step-down board 6, storage battery 7, and API server 8.

[0059] The truss 1 is the hardware placement structure of the device, used to fix and connect other components, ensuring stable and accurate acquisition of images and near-infrared spectra during operation.

[0060] The camera module 2 consists of an ESP32-CAM and an OV2640 140° monocular camera. It is a development system with a built-in Wi-Fi module for the ESP32 chip processor and is compatible with OV series cameras.

[0061] The near-infrared spectral sensor module 3 is a GY-AS7263 spectrometer module and light source. The near-infrared spectral sensor module adopts a small array package of 4.5mm×4.4mm and provides six calibrated spectral channels, which can detect 610nm, 680nm, 730nm, 760nm, 810nm and 860nm.

[0062] The development board 4 for the EPS32 chip is an ESP32-wroom-32 microcontroller development board. The chip has a built-in wireless module that can perform WIFI and Bluetooth transmission. The wireless transmission protocols used include UDP, HTTP, etc. It has multiple I2C ports and serial ports to meet the different data transmission needs of multiple modules.

[0063] The sorting action device 5 is a high-speed airflow device, which is a detachable and optional part, used to sort commercial seedlings through a sorting action.

[0064] The step-down board 6 is a DC-DC adjustable step-down regulated power supply module board with an output voltage requirement of 5V-5.1V and a minimum peak power requirement of 10W.

[0065] The battery 7 is an outdoor power supply battery with a DC output greater than 5V and a peak current carrying capacity greater than 1A.

[0066] The API server 8 is an interface server set up on Huawei Cloud. It is a cloud server using the Linux system, with 1 vCPU group, 4GB of RAM, 40GB of storage, and 2M bandwidth network. The system used is "CentOS_7.3_64bit".

[0067] like Figure 2 As shown, the seedling quality grading and monitoring method based on image and near-infrared spectral phenotypes includes the following steps:

[0068] Step 1: After placing the seedlings on the testing platform, press the "Take Photo" button on the host computer. The host computer will then send a command to the image acquisition interface set up on the API server via the URL, causing the server to start the phenotypic acquisition process, and the device will send a command to acquire the image.

[0069] Step Two: After the device captures an image, it accesses the server's image upload interface and uploads the image. The image is then saved to the server and image distortion correction is performed. The camera's intrinsic and extrinsic parameters are obtained by determining the coordinate points using Zhang Zhengyou's calibration method, and then calculated using the formula r. 2 =x 2 +y 2 Radial distortion correction formula x dr =x(1+k1r) 2 +k2r 4 +k3r 6 ), y dr =y(1+k1r 2 +k2r 4 +k3r 6 ), Tangential distortion correction formula x dt =2p1xy+p2(r 2 +2x 2 )+1, y dt =2p1(r 2 +2x 2 The algorithm )+2p2xy+1 yields a fixed computation matrix, which is used to calculate the corrected image from the input image; where r is the imaging center distance, x dr y dr x dt y dt Let x and y be the coordinates of the image to be corrected, k1, k2, k3, p1, and p2 be the coefficients of the distortion response, and x and y be the corrected coordinates.

[0070] Step 3: The server calls the YOLOv5 model to analyze the image and check if there are actually seedlings in the image to prevent malicious or misuse. If there are no seedlings, an err message will be output to the device, requesting the user to retake the image; otherwise, proceed to the next step.

[0071] Step 4: The server acquires near-infrared spectral data and sends a command to use the near-infrared spectral sensor module to acquire the near-infrared spectral curve of the seedling from the leaf end. The near-infrared spectrometer is then subjected to frequency domain transformation. To reduce the probability of errors in spectral transmission, the Fast Fourier Transform (FFT) method is used to transform the curve into a frequency domain range sequence, which is then saved and received by the server.

[0072] Step 5: The server calls the phenotypic quantization processing command interface to begin quantizing the image phenotypic using image processing algorithms, including:

[0073] S1. Image preprocessing uses Gaussian blur to remove noise from the image. The formula is g(i,j)=h(i,j)*f(i,j), where i and j are the pixel sizes, h(i,j) is the Gaussian smoothing filter set to a 3×3 matrix, f(i,j) are the individual pixels of the original image, and g(i,j) are the individual pixels of the convolutional image. Arranging g(i,j) in the order of the images to form an image matrix will yield the image after noise removal.

[0074] S2. Color enhancement: The formula fin = αG + βB + γR is used, where fin is the enhanced image, α, β, and γ are coefficients, and G, B, and R are the green, blue, and red monochrome channels, respectively. The phenotypic parameters to be extracted include color enhancement. The phenotypic parameters for extracting the plant body are set to α = 1, β = -1, and γ = 0.6, and the phenotypic parameters for extracting withered parts are set to α = -1, β = 0, and γ = 1.

[0075] S3. Phenotypic Mask Extraction: The optimal threshold is obtained using the Otsu method to convert the phenotypic enhanced image into a binary image. The edge fitting of the image is performed using opening and closing operations, and then the color inversion is performed to obtain the mask of the whole seedling and yellow leaves.

[0076] S4. Mask edge detection: This involves applying the Sobel edge detection operator to all masks to extract their edge coordinate matrices. The implementation is described in the formula. Among them G x and G y The image grayscale values ​​after horizontal and vertical edge detection are given, where A is the original image matrix, x and y are the horizontal and vertical coordinates of specific pixels in the image, G is the gradient, and θ is the gradient direction.

[0077] S5. Phenotypic image quantization: Count the number of yellowish patches (masks) to obtain the total number of yellowish spots; accumulate the pixel area of ​​the mask within the closed edges, and then calculate the actual phenotypic area. The function is implemented using the following formula. Where g is the actual area of ​​the mask, f is the number of pixels in the mask, and w 2 The pixel area is typically set to 1, r 2 To map a pixel to its actual area in reality, calculate the leaf area of ​​the entire seedling and the area of ​​each withered yellow spot, and then calculate their ratio, which is the yellow leaf ratio.

[0078] Step Six: The server calls the phenotypic quantization command interface to begin quantizing the near-infrared spectral phenotypic quantification using signal processing algorithms. Based on the varietal characteristics of tomato seedlings, wavelengths of 610nm, 730nm, and 810nm are selected as sampling points, and the calculation formula is used... Where L(w) is the near-infrared spectral curve, L′(w) is the first derivative of the spectral curve, w0 is the x-coordinate of the sampling point, and the first derivative of the sampling point is used as the spectral quantization value.

[0079] Step 7: The server calls the classification command interface to process and classify all the data, and creates a feature value set from leaf area, number of withered yellow spots, yellow leaf ratio, and spectral quantization value. This set is then loaded into the SVM machine learning model for classification to determine the quality of the seedlings.

[0080] Step 8: The server calls the microcontroller driver command interface and sends corresponding drive commands (A, B, and C levels) to the device based on the quality results. This causes the device to operate the sorting drive equipment to perform sorting actions, including: Level A: no high-speed airflow; Level B: medium airflow to propel the seedlings into the second production line; Level C: highest airflow to propel the seedlings into the waste area.

[0081] Step 9: The server calls the table output interface to send the results back to the host computer. The host computer obtains the classification and detection results, and the operation cycle ends.

[0082] like Figure 3 As shown, the training process of the identification model and classification model in the seedling quality grading monitoring method based on image and near-infrared spectral phenotypes includes:

[0083] Step 1: Randomly select a certain number of seedlings and screen them, dividing them into different grade groups and labeling them. In this example, the seedlings are divided into three quality groups: A, B, and C.

[0084] Step 2: Take RGB images of the seedlings from the top of each group to obtain a seedling image sample set.

[0085] Step 3: Select the main seedlings in all the seedling images, mark them with a uniform label, and generate a text file containing the coordinates of the four corners of the selection box and the label.

[0086] Step 4: Match the images and label text files one-to-one to form the dataset, and divide it into training and test sets according to the specified ratio. In this example, the ratio of training to test sets is 7:3.

[0087] Step 5: Set up the training program, using the dataset as input and selecting a training method / model. This example uses the YOLOv5s training method. After training, view the training results report. Evaluation metrics include: accuracy. Accuracy Sensitivity is the same as recall. Specificity TP represents true positives, TN represents false positives, FP represents false positives, and FN represents false positives. All four reference indicators show that higher values ​​indicate better training performance. When precision equals recall, a higher F1 score indicates better training performance. If the training precision is insufficient, increase the number of samples in the dataset; otherwise, proceed to the next step. This model achieves a precision and recall of approximately 1, which gradually stabilizes during training. The final actual recognition rate for seedlings is over 90%, meeting the requirements.

[0088] Step Six: Perform image preprocessing on the seedling image sample set. Use Gaussian blur to remove noise from the images. Use the formula g(i,j)=h(i,j)*f(i,j), where i and j are the pixel sizes, h(i,j) is the Gaussian smoothing filter set as a 3×3 matrix, f(i,j) are the individual pixels of the original image, and g(i,j) are the individual pixels of the convolutional image. Arrange g(i,j) in the order of the images to form an image matrix to obtain the noise-removed image.

[0089] Step 7: Color enhancement. Use the formula fin = αG + βB + γR, where fin is the enhanced image, α, β, and γ are coefficients, and G, B, and R are the green, blue, and red monochrome channels, respectively. Enhance the colors of the phenotypes to be extracted. The phenotypic parameters for extracting the plant body are set to α = 1, β = -1, and γ = 0.6, and the phenotypic parameters for extracting withered parts are set to α = -1, β = 0, and γ = 1.

[0090] Step 8: Phenotypic Mask Extraction. The optimal threshold is obtained using the Otsu method to convert the phenotypic enhanced image into a binary image. The edge fitting of the image is performed using opening and closing operations, and then the color inversion is performed to obtain the mask of the whole seedling and yellow leaves.

[0091] Step 9: Mask edge detection. This involves performing Sobel edge detection on all masks to extract their edge coordinate matrices. The implementation is described by the following formula. Among them G x and G y The image grayscale values ​​after horizontal and vertical edge detection are given, where A is the original image matrix, x and y are the horizontal and vertical coordinates of specific pixels in the image, G is the gradient, and θ is the gradient direction.

[0092] Step 10: Phenotypic image quantization. Count the number of yellowish patches to obtain the total number of yellowish spots; accumulate the pixel area of ​​the mask within the closed edges, and then calculate the actual phenotypic area. The function is implemented using the following formula. Where g is the actual area of ​​the mask, f is the number of pixels in the mask, and w2 The pixel area is typically set to 1, r 2 To map a pixel to its actual area in reality, calculate the leaf area of ​​the entire seedling and the area of ​​each withered yellow spot, and then calculate their ratio, which is the yellow leaf ratio.

[0093] Step 11: Use the detection device of the present invention to acquire the near-infrared spectrum of the seedling leaves.

[0094] Step 12: Perform first-order guided spectrum calculation. In this example, based on the characteristics of the tomato seedling variety, wavelengths of 610nm, 730nm, and 810nm are selected as sampling points, and the calculation formula is used. Where L(w) is the near-infrared spectral curve, L′(w) is the first derivative of the spectral curve, w0 is the x-coordinate of the sampling point, and the first derivative of the sampling point is used as the spectral quantization value.

[0095] Step 13: Use the quantification results and grading of each plant sample as a one-to-one correspondence to create a dataset, and divide it into training and testing sets according to the ratio. In this example, the ratio of training to testing sets in the dataset is 8:2.

[0096] Step Fourteen: Import the dataset using the program and train the SVM machine learning model. After training, you will obtain the trained model and its accuracy. The formula for calculating the training accuracy is provided. Where Acc represents accuracy, Right represents the number of correctly classified samples in the test set, and Total represents the number of samples in the test set. When Acc is lower than expected, samples that are prone to misclassification in the training set need to be added, i.e., supplemented with image and spectral data to quantify the phenotype and expand the dataset, until the trained model reaches the expected level; otherwise, proceed to the next step. In this example, the final accuracy is 95%, which meets the requirements.

[0097] Step 15: Save the model files obtained after training YOLOv5 and SVM and load them for use.

[0098] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the scope of protection of the present invention in any way, and all technical solutions obtained by equivalent substitution or other means fall within the scope of protection of the present invention.

[0099] All parts not covered in this invention are the same as or can be implemented using existing technologies.

Claims

1. A method for monitoring seedling quality grading based on image and near-infrared spectral phenotypes, characterized in that... Includes the following steps: Step 1: Train the recognition model using the YOLOv5s training method; train the classification model using SVM machine learning; the training process of the recognition model is as follows: Step 101: Randomly select a certain number of seedlings and screen them, dividing them into three different quality groups: A, B, and C; Step 102: Take RGB images of the seedlings from the top of each group to obtain a seedling image sample set; Step 103: Select the main body of the seedlings in all the seedling images, mark them with a uniform label, and generate a text file containing the coordinates of the four corners of the selection box and the label; Step 104: Match the images and label text files one-to-one as the dataset, and divide it into two parts, training set and test set, according to the ratio; Step 105: Set up the training program, take the dataset as input, and select the YOLOv5s training method to train the model; after training, check the training result report until the actual seedling recognition rate reaches more than 90%. The learning and training process of the classification model is as follows: Step 106: Perform image preprocessing on the seedling image sample set obtained in Step 102; Step 107: Enhance the color of the sample set; Step 108: Extract phenotypic masks from the sample set; Step 109: Perform phenotypic mask edge detection on the sample set; Step 110: Quantize the phenotypic images of the sample set; Step 111: Obtain the near-infrared spectrum of the seedling leaves; Step 112: Perform first-order guide spectrum calculation, calculation formula: Where: L(w) is the near-infrared spectral curve, L′(w) is the first derivative of the spectral curve, w0 is the x-coordinate of the sampling point, and the first derivative of the sampling point is used as the spectral quantization value. Step 113: Take the quantification results and classification of each plant sample as samples, make a dataset, and divide it into training set and test set according to the proportion. Step 114: Import the dataset using the program and perform SVM machine learning training. After learning, obtain the trained model and model accuracy until the accuracy reaches more than 95%, which meets the requirements. Step 115: Save the model files obtained after training YOLOv5 and SVM and load them for use; The image preprocessing uses Gaussian blur to remove noise from the image. The formula is: g(i,j)=h(i,j)*f(i,j). Arranging g(i,j) in the order of the image to form an image matrix will yield the image after noise removal. Here, i and j are the pixel sizes, h(i,j) is the Gaussian smoothing filter, set to a 3×3 matrix, f(i,j) are the pixels of the original image, and g(i,j) are the pixels of the convolved image. The color enhancement uses the formula: fin=αG+βB+γR, to enhance the color of the phenotype to be extracted, where fin is the enhanced image, α, β, and γ are coefficients, and G, B, and R are the green, blue, and red monochrome channel images, respectively. The phenotypic parameters for extracting the plant body are set to α=1, β=-1, and γ=0.6, and the phenotypic parameters for extracting the withered parts are set to α=-1, β=0, and γ=1. The phenotypic mask extraction uses Otsu's method to obtain the optimal threshold, converting the phenotypic enhanced image into a binary image, and then uses opening and closing operations to fit the edges of the image. Finally, color inversion is performed to obtain the mask for the entire seedling and yellow leaves. The phenotypic mask edge detection involves applying the Sobel edge detection operator to all masks to extract the edge coordinate matrix of the mask. The function is implemented using the following formula: Among them, G x and G y The image grayscale values ​​after horizontal and vertical edge detection, respectively, A is the original image matrix, x and y are the horizontal and vertical coordinates of specific pixels in the image, G is the gradient, and θ is the gradient direction; The phenotypic image is quantized by counting the number of yellowish patches (masks) to obtain the total number of yellowish spots; the pixel area of ​​the mask is calculated by accumulating the pixels within the closed edges, and then the actual phenotypic area is calculated. The function is implemented using the following formula: Where g is the actual area of ​​the mask, f is the number of pixels in the mask, and w 2 r is the pixel area, usually set to 1. 2 To map a pixel to the actual area size in reality, calculate the leaf area of ​​the entire seedling and the area of ​​each withered yellow spot, and then calculate their ratio, which is the yellow leaf ratio. Step 2: After placing the seedlings on the testing platform, press the "Take Photo" button on the host computer to send a command to acquire the image; Step 3: After taking a picture, access the server's image upload interface and upload the picture. Then the picture is saved to the server and image distortion correction is performed. Step 4: The server calls the YOLOv5 recognition model to analyze the image and check whether there are actually seedlings in the image to prevent malicious or incorrect use. If there are no seedlings, an err message will be output to the device, requesting the user to retake the image; otherwise, proceed to the next step. Step 5: The server sends a command to use the near-infrared spectral sensor module to acquire the near-infrared spectral curve of the seedling from the leaf end, and performs frequency domain transformation on the near-infrared spectrometer. The curve is then converted into a frequency domain range sequence using the Fast Fourier Transform (FFT) method, and then the server saves and receives the data. Step 6: The server calls the phenotypic quantization processing command interface to start the image processing algorithm to quantize the image phenotypic quantization. Step 7: The server calls the phenotypic quantization processing command interface to start the signal processing algorithm to quantize the near-infrared spectral phenotypic quantization. Step 8: The server calls the classification command interface to process and classify all the data, loads it into the classification model after SVM machine learning, and obtains the quality status of the seedlings. Step 9: The server calls the driver command interface and sends the corresponding driver commands for the three levels, A, B and C, to the device according to the quality results, so that the device can operate the classification driver to perform classification actions. Step 10: The server calls the tabular output interface to send the results back to the host computer, which then obtains the classification and detection results.

2. The seedling quality grading and monitoring method based on image and near-infrared spectral phenotypes according to claim 1, characterized in that, The image distortion correction method is as follows: The intrinsic and extrinsic parameters of the camera are obtained by measuring the coordinate points using the Zhang Zhengyou calibration method, and then calculated using the formula: r 2 =x 2 +y 2 Radial distortion correction formula: x dr =x(1+k1r) 2 +k2r 4 +k3r 6 ), y dr =y(1+k1r) 2 +k2r 4 +k3r 6 ), Tangential distortion correction formula: x dt =2p1xy+p2(r 2 +2x 2 )+1, y dt =2p1(r 2 +2x 2 The algorithm )+2p2xy+1 yields a fixed computation matrix, which is used to calculate the corrected image from the input image; where r is the imaging center distance, x dr y dr x dt y dt Let x and y be the coordinates of the image to be corrected, k1, k2, k3, p1, and p2 be the coefficients of the distortion response, and x and y be the corrected coordinates.

3. A seedling quality grading and monitoring device based on image and near-infrared spectral phenotypes, used to implement the method as described in claim 1 or 2, characterized in that, Includes a truss, camera module, near-infrared spectral sensor module, development board, step-down board, battery, and API server; among which: The truss serves as the hardware placement structure for the device; The camera module is fixed directly below the truss beam and is used to take pictures of rice seedlings and upload the image data to the server via the network. The near-infrared spectral sensor module is located inside the truss support arm and is used to obtain the near-infrared spectrum of the light reflected from the seedling leaf surface from the side of the seedling. The development board serves as the control center, used for communication, sending commands, receiving data from modules, and transmitting data to the server. The step-down plate is used to provide the device with a suitable operating voltage; The battery is used to power the device when it is operating outdoors; The API server is an interface server located in the cloud, used to exchange data and run program algorithms through interface access.

4. The seedling quality grading and monitoring device based on image and near-infrared spectral phenotype according to claim 3, characterized in that, The monitoring device also includes a classification action device, which serves as a seedling quality screening drive mechanism for grading and then eliminating seedlings.

Images