Sowing quality detection method based on image stitching algorithm and clustering analysis

Abstract

The invention belongs to the technical field of smart agriculture and informatization agriculture, and particularly relates to a seeding quality detection method based on an image splicing algorithm and clustering analysis, namely a rice hole diameter and hole distance measurement method based on a'speed-frame rate 'image splicing mode and a Bayesian Gaussian mixture model. The method comprises the following steps: splicing single seed hole images by using a'speed-frame rate 'to obtain a seedbed panorama, then clustering seed holes by using a Bayesian Gaussian mixture model, and carrying out feature extraction on the clustered seeds in each hole to obtain the maximum circumcircle of the seed holes, thereby calculating the hole diameter of the seeds in each hole. And after the circumcircle of the seeds in each hole is obtained, calculating the pixel distance of the hole distance by using the centroids of the seeds in two adjacent holes. And the actual hole diameter and hole distance are obtained according to the conversion of the pixel coordinates and the world coordinates. According to the method, the problem that the adjacent cave distance cannot be measured in a single-frame image is solved, and compared with a video stream, the image detection method is more efficient and accurate.

Description

Technical Field

[0001] The present invention belongs to the field of smart agriculture and information-based agricultural technology, and specifically relates to a sowing quality detection method based on an image stitching algorithm and cluster analysis, namely, a method for measuring rice hole diameter and hole spacing based on a "speed-frame rate" image stitching algorithm and a Bayesian-Gaussian mixture model clustering algorithm. Background Art

[0002] Mechanized rice planting methods include direct seeding, transplanting, and broadcasting (Wang Zaiman et al. 2015). Direct seeding eliminates the traditional seedling raising and transplanting steps and directly sows the germinated rice seeds in the field, reducing labor and reducing costs (Li Zehua et al. 2018). Precision hole direct seeding for rice, a leading example of direct seeding technology, boasts superior seed-saving performance and has been widely promoted and applied in many provinces and cities across China (Zhang Shun et al. 2023). For example, Zhang Guozhong et al. designed two agitating teeth of identical shape but different mounting positions on a seed suction tray, improving the seeding accuracy of a pneumatic seed metering device and achieving significant seed-saving results (Zhang Guozhong et al. 2013). Zhai Jianbo et al. designed a pneumatic hybrid rice precision hole direct seeding device, addressing the flow issues of hybrid rice within the device (Zhai Jianbo et al. 2016). Xing He et al. designed a multi-channel air suction chamber structure that can adjust the number of air channels to accommodate varying seeding rates (Xing He et al. 2019). In order to address the shortcoming that the suction holes of pneumatic seed metering devices cannot adapt to rice seeds of different sizes, Zhang Kaixing et al. designed a double-circular seed suction tray that can adjust the size of the suction holes according to the size of the seeds (Zhang Kaixing et al. 2019). Zang Ying et al. used the diversion principle to design a diversion seed suction tray in the seed metering device.

[0003] The single-grain sowing accuracy of hybrid rice was improved (Zang Ying et al. 2021). The above-mentioned improvement and optimization research on the precision hole direct seeding technology of rice has promoted the improvement of the mechanization level of precision direct seeding of rice.

[0004] With the deepening of seeding technology research, the demand for rapid evaluation of seeding device performance is growing. Currently, seeding device performance testing methods at home and abroad mainly include manual inspection, photoelectric inspection, piezoelectric inspection, capacitance inspection, and machine vision inspection (Chi Peng et al. 2025, Ding et al. 2021). Manual inspection methods rely primarily on the human eye to subjectively evaluate seeding quality. Direct visual inspection of seeding device performance is only applicable when the seeding device rotates at a low speed. The photoelectric effect detection method, at its core, utilizes optical and electronic principles. By installing photoelectric sensors on both sides of the seeding tube, the falling seeds are identified by pulse signals emitted by the sensors during the seeding process. The photoelectric effect detection method has been widely studied and applied due to its advantages such as automation, fast response time, and ability to continuously process large amounts of data. While this method has extremely high accuracy for detecting missed seeds, it cannot count overlapping seeds when double-seeding occurs, such as when two or more seeds fall simultaneously. Compared to the photoelectric detection method, the piezoelectric effect detection method requires ensuring that falling seeds collide with the sensor to obtain a sensor signal. The piezoelectric effect detection method also has difficulty accurately determining the number of seeds when multiple seeds hit the sensor simultaneously. High-speed cameras can clearly reflect the movement of seeds within the seed meter and the seeds falling, and then combine manual or image processing technology to analyze the performance of the seed meter. The greatest advantage of this method is that high-speed cameras can continuously record the movement trajectory of seeds in a very short time. However, this method has disadvantages such as the high price of high-speed cameras, high requirements for the working environment, and long post-experiment processing time. In addition, it requires a high-speed camera and fill light, making it only suitable for indoor research.

[0005] In the prior art, CN116452526A, a rice seed identification and counting method based on image detection, is to evenly sprinkle rice seeds on white grid paper, shoot from above with a camera lens parallel to the grid paper, and then use a deep convolutional neural network to train the captured image. The collected rice seed image is divided into multiple sub-images and fed into a trained classification model. The number of rice seeds corresponding to the classification results of each rice seed image in the original rice seed image is then counted to obtain the total number of rice seeds in the original rice seed image. However, this method cannot simulate the field planting environment in real time, that is, it cannot be actually applied to the real-time detection work of field sowing in a real environment. Moreover, it only counts the number of rice seeds and cannot detect the sowing quality in the actual working environment of the farmland in real time.

[0006] In recent years, with the development of deep learning and industrial camera technology, researchers at home and abroad have gradually applied machine vision to seeding quality inspection. Machine vision inspection offers significant advantages in real-time performance, accuracy, automation, and information collection, potentially improving the performance and intelligence of seed metering devices. Currently, with the rapid development of machine vision technology, more and more seeding inspection technologies are adopting machine vision. Inspection technology is also moving towards intelligentization, improving the accuracy and efficiency of seeding inspection and gradually replacing traditional manual sampling inspection methods, which are labor-intensive, tedious, and inefficient. Summary of the Invention

[0007] In order to detect the quality of rice sowing, especially to measure the diameter and distance of rice holes, the present invention provides an image stitching algorithm based on the "speed-frame rate" linkage. Rice seed hole images are acquired through stitching and a panoramic view of the rice seed bed is generated. The panoramic view can reflect the overall distribution of seeds in the detection area. Then, a clustering algorithm such as the Bayesian Gaussian mixture model (BGMM) is selected to model the rice seeds in each hole. First, an optimized recognition algorithm is used to accurately identify the rice seeds in the hole. Then, the BGMM algorithm is used to cluster the hole to obtain the centroid and circumscribed circle of the hole. The diameter of the circumscribed circle is calculated to obtain the hole diameter. The distance between the centroids of two adjacent holes is calculated to obtain the hole distance, so as to quantitatively analyze the sowing quality.

[0008] The specific technical solution of the present invention is as follows: a sowing quality detection method based on image splicing algorithm and cluster analysis, comprising the following steps:

[0009] S1. Obtain multiple original rice seed hole sample images in the farmland;

[0010] S2, synthesizing sample images of the seed bed panorama based on all original rice seed hole sample images;

[0011] S3, using an image processing model to process the seed bed panorama and identify the centroids of various seeds;

[0012] S4. Use the cluster analysis method to cluster the centroids of the seeds identified in step 3, divide the seeds in the seed bed into holes, determine the centroid of each hole, and calculate the hole diameter within each hole and the hole distance between adjacent holes.

[0013] In step S1, a plurality of original rice seed hole sample images are obtained, which specifically includes the following steps:

[0014] S1.1: As the seed drill of the seed drill moves forward, an industrial camera is used to photograph the farmland to obtain rice sample image data; S1.2: An image database of rice seed holes is constructed.

[0015] The step S1.2 constructs an image database of rice seed holes, specifically comprising the following steps:

[0016] S1.2.1: Preprocessing of rice seed image data to remove irrelevant information from the image background and remove blurred and damaged images;

[0017] S1.2.2: Use the Labelimg annotation tool to annotate the preprocessed rice seed hole images and generate corresponding annotated rice seed hole image sample images;

[0018] S1.2.3: Dataset division: randomly divide the labeled dataset into training and test sets in a ratio of 6:4;

[0019] S1.2.4: Data augmentation processing: During the training process, online data augmentation technology is used to perform real-time transformations on the images in the training set, including image processing technologies such as scaling, random cropping, random rotation, and random horizontal flipping.

[0020] The step S2 synthesizes a sample image of the seed bed panorama based on all the original rice seed hole sample images, specifically comprising the following steps:

[0021] S2.1. Find the overlapping area of ​​each rice seed hole sample image frame relative to the next rice seed hole sample image frame, and use the non-overlapping area as the area to be stitched;

[0022] S2.2. Truncate and remove the overlapping areas of each frame of the rice seed hole sample image except the last frame of the rice seed hole sample image to obtain the area to be stitched, and stitch the area to be stitched to the starting frame side of the rice seed hole sample image of the next frame in the order of frame acquisition time. After stitching all the areas to be stitched, a panoramic view of the seed bed is obtained.

[0023] In step S3, the image processing model is used to process the seed bed panorama to identify the centroids of various seeds, which specifically includes the following steps:

[0024] The spliced ​​image is imported into the YOLOv8 recognition algorithm model to accurately identify the rice seeds in the seed hole and derive the pixel coordinates of the four vertices of each seed. These four vertices belong to the prediction box of the seed. The pixel coordinates of each vertex of the prediction box are obtained, A(y0, x0), B(y0, x1), C(y1, x0), D(y1, x1), and then the pixel coordinates of the seed centroid c (x m ,y m ), as shown in formula (6),

[0025] The step S4 specifically includes the following steps:

[0026] S4.1. Determine the centroid pixel coordinates of each seed hole and use the Bayesian Gaussian mixture model to determine the centroid coordinates of each seed hole (C x , C y );

[0027] S4.2. Calculate the hole diameter and obtain the maximum distance d from all seeds in each hole to the centroid. max , to determine the hole diameter, as shown in formula (7),

[0028]

[0029] S4.3. Calculate the hole distance. First determine the pixel value of the distance between the centers of the circumscribed circles of two adjacent seed holes. Then, in the camera coordinate system, combined with the camera height, calculate the distance between the centers of the circumscribed circles of the seeds of the two adjacent holes. As shown in (8), the actual distance S between the centroids of the two holes is obtained.

[0030]

[0031] Where Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the detection device camera to the seed hole, mm; S is the distance between the detection holes, mm.

[0032] Before calculating the hole diameter and hole distance in step S4, the intrinsic and extrinsic parameter matrices of the camera are obtained, and the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system.

[0033] The conversion formula between the pixel coordinates of the identified seed and the camera coordinates is shown in formula (9).

[0034]

[0035] In the above formula, Represents the pixel coordinates in the image, where x and y are coordinates on the image plane, and 1 is a homogeneous coordinate used to simplify matrix operations;

[0036] is the intrinsic parameter matrix of the camera, du and dv are the scale factors from pixel units to camera coordinate units, x0 and y0 are the coordinates of the principal point of the image (usually the center of the image);

[0037] is the focal length matrix of the camera, where f is the focal length of the camera. This matrix transforms the point in the camera coordinate system into the world coordinate system;

[0038] It is the three-dimensional coordinate in the camera coordinate system. By multiplying these matrices, the three-dimensional coordinate camera coordinate (X a , Y a , Z a );

[0039] is the external parameter matrix, where f u and f v are the components of the focal length in the u and v directions, x0 and y0 are the coordinates of the principal point; M represents the calibrated camera intrinsic parameter matrix.

[0040] Step S4.1 includes:

[0041] (1) Initialization processing

[0042] Assume that the data is generated by a mixture of K Gaussian distributions, and the mean μ of each Gaussian distribution is k Corresponding to the centroid of a cluster;

[0043] Including, initializing the parameters of the Gaussian distribution: the mean μ of each Gaussian distribution k , covariance matrix Σk and weight π k Initialize the data by randomly selecting data points as the mean, initialize the covariance matrix to the unit matrix or the covariance matrix of the data, and the weight π k Initialized to uniform distribution;

[0044] (2) Expectation step: For each seed centroid position x i , calculate the posterior probability r that it belongs to the kth Gaussian distribution ik , the formula is:

[0045]

[0046] p(x|θ k ) is the probability density function of the kth Gaussian distribution, x is the seed centroid position vector, θ k =(μ k ,Σk,π k ) is the parameter of the k-th Gaussian distribution, d is the dimension of the data, is the probability density function of the entire Gaussian mixture model;

[0047] (3) Maximization step: Update the model parameters based on the posterior probability calculated according to the expectation to maximize the log-likelihood function of the observed data.

[0048] Update the mixing weight π k : Where N is the total number of seed centroids;

[0049] Update mean μ k : Among them, x i represents a data point;

[0050] Update the covariance matrix ∑k:

[0051] where · is the data point x i and mean μ k The outer product of the deviations;

[0052] (5) Iteration

[0053] By repeatedly performing the expectation step and the maximization step until the parameters converge;

[0054] (6) Determine the cluster centroid

[0055] Calculate the centroid: After convergence, the centroid of the seed hole is calculated using the mean μ of each Gaussian distribution k According to the weight π k The weighted sum is obtained, that is,

[0056] Compared with the prior art, the beneficial effects of the present invention are: an image stitching method based on the "speed-frame rate" linkage solves the problem that a single-frame image cannot accurately measure the distance between adjacent holes. Compared with video streams, the image detection method is more efficient and accurate. Based on the established rice seed detection model, the Bayesian-Gaussian mixture model is used to divide the seed holes, and the geometric center of each rice seed hole is calculated. Finally, scale verification and bench test verification were carried out, and the hole diameter and hole distance values ​​detected based on the Bayesian-Gaussian mixture model were compared with the manually measured values ​​to obtain the relative average error. The relative average error results show that the hole diameter and hole distance detection can be completed well under laboratory conditions and field environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 This is a schematic diagram of determining the overlapping area of ​​two adjacent frames of rice seed hole sample images in Example 1;

[0058] Figure 2 The method for splicing two adjacent frame images in Example 1;

[0059] Figure 3 This is the seed bed panoramic stitching method in Example 1;

[0060] Figure 4 A schematic diagram of obtaining the coordinates of the center point of the seed prediction box by the YOLOv3 recognition algorithm model in Example 1;

[0061] Figure 5 This is the clustering effect diagram of the Bayesian-Gaussian mixture model in Example 1;

[0062] Figure 6 This is the rice seed clustering effect diagram obtained by the BGMM clustering algorithm in Example 1;

[0063] Figure 7To take chessboard pictures at different angles with a camera;

[0064] Figure 8 Schematic diagram of feature point extraction for chessboard calibration;

[0065] Figure 9 The internal specific parameters of the industrial camera after the camera is calibrated using the Zhang Zhengyou calibration method in Example 1;

[0066] Figure 10 The process of converting the pixel coordinates of rice seeds into camera coordinates in Example 1

[0067] Figure 11 Comparative analysis of the detection values ​​of the cluster analysis model in Example 1 with the values ​​obtained by manual measurement;

[0068] Figure 12 This is the scale verification result in Example 2;

[0069] Figure 13 This is a physical picture of the crawler vehicle stand of Example 2;

[0070] Figure 14 The detection values ​​of the cluster analysis model of Example 2 were compared and analyzed with the values ​​obtained by manual measurement. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0072] Example 1

[0073] This embodiment is a detection method implemented based on a pneumatic rice precision direct seeding machine detection system, which includes an industrial camera, a host computer, a core processor, and a mobile power supply. The pneumatic rice precision direct seeding machine is designed with a camera bracket with freely adjustable detection height based on a slide rail to facilitate adjustment of different detection heights.

[0074] During direct seeding of rice, the field must maintain adequate moisture, creating a complex operating environment. Furthermore, rice seeds are smaller than seeds of crops like corn, placing higher demands on the performance of the detection system. The image acquisition device, the "eyes" of the detection system, significantly impacts its accuracy. Industrial cameras, with their high performance, stability, and reliability, effectively meet the requirements for capturing images of rice seeds in the field during seeding, thereby enhancing the accuracy and stability of the detection system.

[0075] The rice seeds sown by the double-chamber pneumatic precision rice hole seeder have obvious hole formation, with an average of 3 to 5 seeds per hole. The clustering method can be used to analyze the sowing quality.

[0076] By obtaining multiple original rice seed hole sample images and constructing a rice seed hole image database, it is convenient to manage the original rice seed hole sample images, thereby facilitating subsequent image processing and training, and obtaining a neural network seed hole recognition model with a higher recognition rate; by processing all original rice seed hole sample images in the rice seed hole image database, it is convenient to obtain a high-quality target rice data set, providing samples for subsequent deep learning methods, thereby facilitating subsequent deep learning training, effectively improving the training quality, and thus improving the accuracy of the neural network rice ear recognition model in locating and identifying the rice ears to be detected.

[0077] This embodiment mainly carries out the quality inspection of rice seed sowing based on the indicators such as the number of seeds in the holes, hole diameter, hole distance, and hole formation. The specific steps are as follows:

[0078] S1. Obtain multiple original rice seed hole sample images in the farmland;

[0079] S2, synthesizing sample images of the seed bed panorama based on all original rice seed hole sample images;

[0080] S3, using the image processing model YOLOv3 deep learning method to process the seed bed panorama and identify the centroid of each seed;

[0081] S4. Use the cluster analysis method to cluster the centroids of the seeds identified in step 3, divide the seeds in the seed bed into holes, determine the centroid of each hole, classify the seed hole groups with different densities, and calculate the hole diameter within each hole and the hole distance between adjacent holes.

[0082] Wherein, in step S1, a plurality of original rice seed hole sample images are obtained, which specifically includes the following steps:

[0083] S1.1: When the seed drill of the seed drill moves forward, the farmland is photographed using an industrial camera to obtain rice sample image data.

[0084] Preferably, in this step, the forward speed of the direct seeding machine is 0.2 m / s, the rotation speed of the seeding shaft of the seeding device is 25 r / min, and the camera frame rate is 24.2.

[0085] Preferably, in this step, a CMOS industrial camera with a GigE Vision data line interface is used, and the model is Hikvision MV-CS050-10GC.

[0086] S1.2: Constructing a rice seed hole image database and a rice seed hole recognition model, specifically including the following steps:

[0087] S1.2.1: Preprocessing of rice seed image data to remove irrelevant information from the image background and remove blurred and damaged images.

[0088] Given that the acquired image background carries a large amount of irrelevant information and has a certain impact on the overall detection accuracy and processing speed, preprocessing is performed on the original dataset to eliminate unnecessary interference, thereby improving the accuracy and speed of model recognition detection in the next step.

[0089] S1.2.2: Use the Labelimg annotation tool to annotate the pre-processed rice seed hole image and generate the corresponding annotated rice seed hole image sample image. The annotation content includes the location of the seed hole (bounding box), etc.

[0090] S1.2.3: Dataset Partitioning: Randomly divide the labeled dataset into training and test sets in a ratio of 6:4. Ensure that the distribution of the training and test sets is representative, which can effectively enhance the generalization performance of the model and avoid information from the test set being carried over into the training set.

[0091] S1.2.4: Data enhancement processing: Use online data enhancement technology during the training process to perform real-time transformations on the images in the training set. This specifically includes the following:

[0092] Zoom: Adjust the size of the image to simulate images at different shooting distances.

[0093] Random cropping: Randomly cropping part of the image to increase the model's adaptability to local features.

[0094] Random rotation: Randomly rotate the image by a certain angle to enhance the robustness of the model to different shooting angles.

[0095] Random Horizontal Flip: Randomly flip the images horizontally to further enrich the dataset.

[0096] Therefore, when establishing the image database of rice seed holes, the first step is preprocessing, the purpose of which is to clean up the data and remove irrelevant information, which is the basis for subsequent labeling and model training. If labeling is performed first and then preprocessing is performed, the labeling information may be lost or inaccurate. The second step is to label and divide the preprocessed images. Labeling comes before division because labeling is performed on the original or preprocessed images, and the data set can only be divided after labeling is completed. The data set is divided to ensure the independence of training and testing and to avoid data leakage. The third step is to enhance the data. Data enhancement is performed in the training stage: data enhancement is performed on the training set, and the purpose is to dynamically generate more varied images during the training process and enhance the generalization ability of the model. The test set does not need data enhancement to ensure the objectivity of the test results. The above steps construct a rich data set to prepare for the next step of improving the generalization ability of the model.

[0097] In the above step S2, the seed bed panoramic image is obtained by an image stitching algorithm based on the planter movement speed and the camera frame rate, including the following specific steps:

[0098] S2.1. Find the overlapping area of ​​each frame of rice seed hole sample image relative to the next frame of rice seed hole sample image, and use the non-overlapping area as the area to be stitched.

[0099] Taking the rice seed hole sample image obtained in step S1.1 as the object, the rice seed hole sample image accurately shows the overall distribution of seeds in the field. Multiple single-frame seedbed images are used to stitch together a rice seed panorama, thereby visualizing the seed distribution of the entire detection area. To stitch the panorama, we must first find the overlapping parts of the image stitching area. Through analysis, we can know that the camera's field of view will change with the movement of the planter, and the seed holes captured at time t1 and t2 are different. Figure 1 As shown, it can be seen that there is an overlapping area between the two frames of image, and the overlapping area is represented by O1.

[0100] The forward speed of the planter is set to V1, the camera moves along with the planter, the frame rate of image acquisition is f1, and the size of a single frame image is h1×w1, where h1 is the height of a single frame image and w1 is the width of a single frame image. Figure 1 As shown in Figure 1, during the acquisition time interval t between two adjacent frames, the camera's field of view moves along with the planter by a distance l1, and the size of the non-overlapping region between the two adjacent frames is h1 × l1. The time interval t between two adjacent images can be calculated using the image acquisition frame rate in Equation 1. The field of view movement distance l1 corresponding to each frame during image acquisition can be calculated using Equation 2. Therefore, the overlapping region width l2 is calculated using Equation 3:

[0101]

[0102] S2.2. Truncate and remove the overlapping areas of each frame of the rice seed hole sample image except the last frame of the rice seed hole sample image to obtain the area to be stitched, and stitch the area to be stitched to the starting frame side of the rice seed hole sample image of the next frame in the order of frame acquisition time. After stitching all the areas to be stitched, a panoramic view of the seed bed is obtained.

[0103] To better illustrate the image stitching process, we first take the stitching of two images as an example, and then introduce the general seed bed panoramic stitching based on this. Figure 2 It is a method for stitching two adjacent frame images.

[0104] First, the overlapping area in the previous frame is identified and then the overlapping area in the previous frame is cut off. The non-overlapping area is retained and stitched to the start frame side of the next frame in the order of frame acquisition time ( Figure 3 (The left side of the next frame is in the middle). Finally, the frames are aligned to form a stitched image. The height h2 and width w2 of the stitched image are calculated as shown in Equation 4.

[0105]

[0106] like Figure 3 Figure 1 shows a method for stitching a panoramic seedbed image. First, the overlapping regions of each image frame are truncated, retaining the non-overlapping regions. Then, based on the planter's direction of travel and the order in which each frame was acquired, all non-overlapping regions are stitched together. The seedbed panorama has a height of H1 and a width of W1. The equations for H1 and W1 are shown in Equation 5.

[0107] m is the number of frames.

[0108] In this step, a panoramic view of the seed bed is obtained for each shot taken by the industrial camera, thereby forming a sample image of the panoramic view of the seed bed for use in the training and learning of the image processing model in the next step.

[0109] In step S3, the seedbed panorama obtained in step S2 is imported into the image recognition model to obtain the centroid of each seed. In this step, the trained YOLOv8 image recognition model is used to extract features from the sample images of the target seedbed panorama. The sample images of the entire seedbed panorama contain many rice seeds that occupy a small area of ​​the entire image and are scattered and poorly distributed. The YOLOv8 image recognition model was trained using the multiple original rice seed hole sample images from step 1.

[0110] The spliced ​​image is imported into the trained YOLOv8 image recognition model to accurately identify the rice seeds in the seed hole and derive the pixel coordinates of the four vertices of each seed. These four vertices belong to the seed prediction box. The forward direction of the direct seeding machine is the y-axis, and the direction perpendicular to the forward direction is the x-axis. The coordinates of each vertex of the seed prediction box are set as A(y0, x0), B(y0, x1), C(y1, x0), and D(y1, x1). Figure 4 As shown. The pixel coordinates (x m ,y m ).

[0111]

[0112] After using YOLOv8 to accurately identify the centroids of each seed in a hole, the next step is to perform cluster analysis on the centroids of the seeds in each hole. Therefore, in step S3, using the trained YOLOv8 to perform feature extraction on the target rice seedbed panoramic image dataset ensures that each rice seed hole can be effectively clustered, thereby accurately obtaining the cluster center of the seed hole.

[0113] For single seeds, the cluster analysis process in step S4 requires only detecting the pixel coordinates of the centroids of two adjacent seeds, calculating the Euclidean distance between the centroids, and converting this distance into the actual distance. However, for hole-sown rice, where each hole contains multiple seeds, it is necessary to first obtain the centroids of each seed in that hole. Cluster analysis is then performed using a Bayesian-Gaussian mixture model to determine the centroid of the hole, and then calculate the hole diameter and the distance between adjacent seeds in the hole.

[0114] The Bayesian Gaussian Mixture Model (BGMM) is a clustering method based on Bayesian theory. It improves the traditional Gaussian Mixture Model (GMM) by introducing the Bayesian framework. BGMM can automatically determine the optimal number of clusters. Its clustering effect is as follows: Figure 5 In addition, BGMM automatically determines the optimal number of clusters through Bayesian inference, avoiding the limitation of the KMeans clustering algorithm and AHC's bottom-up hierarchical clustering method that requires the pre-specified number of clusters. At the same time, it overcomes the parameter sensitivity problem of DBSCAN, a clustering algorithm that uses regional density.

[0115] The method of determining the centroid of each cluster by Bayesian Gaussian mixture model (BGMM) mainly includes the following steps:

[0116] (1) Initialization processing steps (model assumptions)

[0117] Select the amount of Gaussian distribution data and determine the number of Gaussian distributions in BGMM, that is, the number of clusters. Assume that the data is generated by a mixture of K Gaussian distributions, and the mean μ of each Gaussian distribution is k Corresponds to the centroid of a cluster.

[0118] Including, initializing the parameters of the Gaussian distribution: the mean μ of each Gaussian distribution k , covariance matrix Σk and weight π k Initialize. Usually, randomly select data points as the mean, initialize the covariance matrix to the unit matrix or the covariance matrix of the data, and the weight π k Initialized to a uniform distribution.

[0119] (2) Expectation step: For each seed centroid position x i , calculate the posterior probability r that it belongs to the kth Gaussian distribution ik , the formula is:

[0120]

[0121] p(x|θ k ) is the probability density function of the kth Gaussian distribution, x is the seed centroid position vector (x = (x, y) in the two-dimensional plane), θ k =(μ k ,Σk,π k ) is the parameter of the k-th Gaussian distribution, d is the dimension of the data, is the probability density function of the entire Gaussian mixture model;

[0122] (3) Maximization step (M step): The goal of this step is to update the model parameters based on the expected posterior probability calculated to maximize the log-likelihood function of the observed data.

[0123] Update the mixing weight π k : where N is the total number of seed centroids.

[0124] Update mean μ k : Among them, x i Represents a data point.

[0125] Update the covariance matrix ∑k:

[0126] Among them, (x i -μ k )(x i -μ k ) T is the data point x i and mean μ kThe outer product of the deviations.

[0127] (5) Iteration

[0128] The parameters of the BGMM can be optimized by repeatedly executing the expectation step and the maximization step until the parameters converge (that is, the change in the parameters is less than a preset threshold or the maximum number of iterations is reached). In each iteration, the E step provides the necessary statistical information for the M step, and the M step uses this information to update the model parameters, thereby gradually improving the model's fit to the data.

[0129] (6) Determine the cluster centroid

[0130] Calculate the centroid: After BGMM converges, the mean μ of each Gaussian distribution i can be considered as the centroid of the corresponding cluster.

[0131] The centroid of the seed hole can be expressed as the mean μ of each Gaussian distribution k According to the weight π k The weighted sum is obtained, that is,

[0132] Step S4 specifically includes the following steps:

[0133] S4.1. Obtain the centroid pixel coordinates of each seed hole: Use the Bayesian Gaussian mixture model to obtain the centroid coordinates of each seed hole (C x , C y ), in this step, after the clustering operation is completed, the clustering results are deeply analyzed, focusing on the rationality and stability of the clustering. After running the clustering algorithm, the model directly outputs the clustering effect diagram, which facilitates a clear understanding of the data distribution characteristics. In order to further verify the performance of the BGMM model, random samples are collected from the sample image set, and the trained BGMM model is used to identify and detect the extracted samples. The rice seed clustering effect obtained by the BGMM clustering algorithm is shown in the figure below. Figure 6 The visual clustering display shows that BGMM can accurately and stably divide rice seeds in each hole, laying the foundation for the subsequent measurement of hole diameter and hole distance.

[0134] S4.2. Calculate the hole diameter. For each seed hole, calculate a minimum circumscribed circle that contains all seeds.

[0135] The calculation of the pixel value of the hole diameter is performed using formula (7):

[0136]

[0137] In the formula, (x i ,y i ) is the pixel coordinate of the i-th seed in the hole; (C x , C y) are the pixel coordinates of the centroid of the seed hole. n is the total number of seeds.

[0138] The hole diameter is the diameter of the circumscribed circle containing the most seeds in each hole, which can be calculated by calculating the maximum distance d from all seeds to the centroid. max to confirm.

[0139] S4.3. Calculate the hole distance. First, determine the pixel value of the distance between the centers of the circumscribed circles of two adjacent seed holes. Then, in the camera coordinate system, calculate the distance between the centers of the circumscribed circles of the two adjacent seed holes, as shown in formula (8). This formula calculates the actual distance between the centers of the two holes.

[0140]

[0141] Where Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the detection device camera to the seed hole, mm; S is the distance between the detection holes, mm.

[0142] In step S4, the pixel coordinates are converted to three-dimensional coordinates in the camera coordinate system using the camera's intrinsic and extrinsic parameters. This step is usually performed before calculating the hole diameter and hole distance in step S4, because the position of the seeds in the camera coordinate system is required to further calculate the actual distance between them.

[0143] In step S3, after obtaining the pixel coordinates of each seed in the rice seed hole and the pixel coordinates of the center of mass of the seed hole, the working space of the camera is defined as the camera coordinate system, and the pixel coordinates (x, y) are converted into camera coordinates (x a , Y a , Z a ) and perform coordinate transformation.

[0144] When calibrating the camera, the problem that the height of the chessboard used is inconsistent with the actual height of the rice seeds can be solved by rigid transformation. Camera calibration is crucial for the visual system and is the first step to convert pixel distance into actual distance. After selecting the camera model, the Zhang Zhengyou calibration method is used to calibrate the camera to obtain the camera's internal parameter matrix. The chessboard size selected is 16×13, and the black and white square size is 10mm. Then use the camera to take more than 20 pictures of the chessboard in different postures, such as Figure 7 shown.

[0145] After importing the captured image into Matlab, the inner corners of the chessboard are detected using the built-in feature point extraction algorithm to obtain the pixel coordinates of the corresponding points, such as Figure 8 As shown in the figure, the inner corner points of the chessboard are extracted. After calibrating the camera using Zhang Zhengyou's calibration method, the calculated calibration error is 0.14 pixels. The specific internal parameters of the camera are as follows: Figure 9The process of converting rice seed pixel coordinates into camera coordinates is as follows. Figure 10 shown.

[0146] Depend on Figure 10 It can be seen that the conversion formula between the pixel coordinates of the identified seed and the camera coordinates is shown in formula (9).

[0147]

[0148] In the above formula, Represents the pixel coordinates in the image, where x and y are coordinates on the image plane and 1 is a homogeneous coordinate to simplify matrix operations.

[0149] is the intrinsic parameter matrix of the camera, du and dv are the scale factors from pixel units to camera coordinate units, x0 and y0 are the coordinates of the principal point of the image (usually the center of the image).

[0150] Is the focal length matrix of the camera, where f is the focal length of the camera. This matrix transforms points in the camera coordinate system into the world coordinate system.

[0151] is the three-dimensional coordinate in the camera coordinate system. By multiplying these matrices, we can get the three-dimensional coordinate camera coordinate (X a , Y a , Z a ).

[0152] is the external parameter matrix, where f u and f v are the components of the focal length in the u and v directions, and x0 and y0 are the coordinates of the principal point.

[0153] Where: M represents the camera intrinsic parameter matrix that has been calibrated, such as Figure 9 As shown in .

[0154] The pixel coordinates in the image are converted to three-dimensional coordinates in the camera coordinate system through the camera's intrinsic and extrinsic parameter matrices. Through the above transformation, the camera coordinates corresponding to the pixel coordinates of each rice seed in the seed hole can be obtained.

[0155] This conversion is crucial for precise positioning and manipulation in areas such as robotic vision systems and agricultural automation. In this way, the two-dimensional information in the image can be converted into information in three-dimensional space, enabling accurate recognition and manipulation of objects.

[0156] After the above transformation, the camera coordinates of the rice seeds in the image are obtained, allowing the actual seed hole diameter and the distance between adjacent seed holes to be calculated. First, the circular coordinates of the circumscribed circle of each seed hole are determined. Then, using the Euclidean formula (as shown in Equation 10), the distance between the centers of the two circumscribed circles is calculated to obtain the actual seed hole distance.

[0157]

[0158] Where: (X1, Y1) and (X2, Y2) are the coordinates of the centers of two adjacent circumscribed circles.

[0159] S4.3. Verify the recognition accuracy of the measurement of the diameter of the rice seed holes and the distance between adjacent seed holes in the cluster analysis model in the above steps.

[0160] In order to verify whether the above image stitching, recognition, clustering and conversion process has practical application value, 10 sample hole images were selected from the collected seed hole images, the pixel coordinates of the rice seeds were converted into camera coordinates, and the actual hole diameter and hole distance values ​​were calculated. The image stitching part is shown in Figure 2. Figure 2 and 3 Then the detection value based on the model is compared with the value obtained by manual measurement, and the error rate is calculated respectively. The results are shown as follows: Figure 11 shown.

[0161] To judge the recognition accuracy of the clustering model in step 4, Figure 11 The hole diameter and hole distance detection values ​​are judged based on the degree of proximity of the mean relative error (MRE) value to 0. MRE represents the relative average error between all model detection values ​​and manual measurements and is used to measure the average error of model detection values ​​relative to manual measurements. The closer the MRE value is to 0, the higher the detection accuracy of the detection system, that is, the closer the model detection value is to the manual measurement value. The expression for mean relative error is shown in (4-10):

[0162]

[0163] Where: e represents the MRE value; n is the number of test holes; y i is the manual detection value of the i-th acupoint; is the model detection value of the i-th sample.

[0164] The calculated MRE values ​​for hole diameter and hole spacing were 0.67 and 0.66, respectively, both approaching 0. This indicates that the Bayesian-Gaussian mixture model-based measurement results are relatively close to the manual measurements, confirming the feasibility of the cluster analysis method in step 4 for seeding quality detection.

[0165] Example 2

[0166] In order to verify the effectiveness of the present invention, the clustering in step 4 in Example 1 is verified.

[0167] 1. Ruler verification

[0168] To further verify the feasibility of the cluster analysis model detection method in step 4 of the above embodiment, three sets of recognition verification were carried out at different camera heights. The industrial camera was installed on the test bench at 300mm, 400mm, and 500mm heights, and then the rice seeds at the 0 scale and 40 scale on the ruler were photographed. The model was used for recognition detection. The model detection value results were as follows: Figure 12 shown.

[0169] Depend on Figure 12 It can be seen that the deviation between the detected distance and the actual value is within 3 mm, which proves that the detection method is relatively accurate and has certain practical significance and can be used to detect rice sowing quality.

[0170] 2. Bench test verification

[0171] Considering that the external conditions in the laboratory environment have little interference with the detection system, in order to be closer to the field detection environment, a set of intelligent crawler rice sowing device with single-row sowing was built. The sowing device consists of a double-chamber air-suction rice seeding device, a negative pressure machine, a motor, a motor speed control board, a camera, a computer, and a crawler vehicle. The specific structure is as follows Figure 13 shown.

[0172] The sowing device's motor speed is adjustable using an STM32F103ZET6 microcontroller. The sowing pressure is fixed at -8 kPa, and the forward speed of the sowing machine can be adjusted via a computer, with settings of 0.1 m / s, 0.2 m / s, and 0.3 m / s. The experimental plot was located on the campus of Huazhong Agricultural University, with sowing time set at 3:00 PM in sufficient sunlight. Before the experiment began, the motor speed was adjusted to 30 r / min and the forward speed of the crawler vehicle to 0.1 m / s. A 3-meter buffer zone was reserved. Once the sowing performance of the sowing machine stabilized, video recording began. After recording, each frame of the video was imported into the detection system, which outputted the test data. During the experiment, manual measurement was used to measure the number of seeds per hole, hole diameter, and hole spacing. The experiment was divided into three groups, with 150 holes measured in each group and the average values ​​calculated.

[0173] Then the detection results of the detection system are compared with the manual detection results, as follows Figure 14 As shown in the figure, the average error rate of the system in detecting the hole distance is 5.50%, and the average error of the hole diameter is 13.25%.

[0174] right Figure 14The hole diameter and hole distance detection values ​​were also analyzed using the relative mean error (MRE) method. The final MRE value for hole diameter was 0.78, and the MRE value for hole distance was 0.84. The MRE values ​​for hole diameter and hole distance were both close to 0. The relative mean error results show that the detection value based on the Bayesian-Gaussian mixture model in step 4 of Example 1 is close to the manual measurement value, which proves that this method is also feasible for sowing quality detection in a field environment.

[0175] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A sowing quality detection method based on image splicing algorithm and cluster analysis, characterized in that: The following steps are included: S1. Obtain multiple original rice seed hole sample images in the farmland; S2, synthesizing sample images of the seed bed panorama based on all original rice seed hole sample images; S3, using an image processing model to process the seed bed panorama and identify the centroids of various seeds; S4. Use the cluster analysis method to cluster the centroids of the seeds identified in step 3, divide the seeds in the seed bed into holes, determine the centroid of each hole, and calculate the hole diameter within each hole and the hole distance between adjacent holes.

2. The sowing quality detection method based on image splicing algorithm and cluster analysis according to claim 1, characterized in that: In step S1, a plurality of original rice seed hole sample images are obtained, which specifically includes the following steps: S1.1: As the seed drill of the seed drill moves forward, an industrial camera is used to photograph the farmland to obtain rice sample image data; S1.2: An image database of rice seed holes is constructed.

3. The method for detecting rice sowing quality based on image splicing algorithm and cluster analysis according to claim 2, characterized in that: The step S1.2 constructs an image database of rice seed holes, specifically comprising the following steps: S1.2.1: Preprocessing of rice seed image data to remove irrelevant information from the image background and remove blurred and damaged images; S1.2.2: Use the Labelimg annotation tool to annotate the preprocessed rice seed hole images and generate corresponding annotated rice seed hole image sample images; S1.2.3: Dataset division: randomly divide the labeled dataset into training and test sets in a ratio of 6:4; S1.2.4: Data augmentation processing: During the training process, online data augmentation technology is used to perform real-time transformations on the images in the training set, including image processing technologies such as scaling, random cropping, random rotation, and random horizontal flipping.

4. The sowing quality detection method based on image splicing algorithm and cluster analysis according to claim 1 is characterized in that: The step S2 synthesizes sample images of the seed bed panorama based on all original rice seed hole sample images, specifically comprising the following steps: S2.

1. Find the overlapping area of ​​each rice seed hole sample image frame relative to the next rice seed hole sample image frame, and use the non-overlapping area as the area to be stitched; S2.

2. Truncate and remove the overlapping areas of each frame of the rice seed hole sample image except the last frame of the rice seed hole sample image to obtain the area to be stitched, and stitch the area to be stitched to the starting frame side of the rice seed hole sample image of the next frame in the order of frame acquisition time. After stitching all the areas to be stitched, a panoramic view of the seed bed is obtained.

5. The sowing quality detection method based on image splicing algorithm and cluster analysis according to claim 1 is characterized in that: In step S3, the image processing model is used to process the seed bed panorama to identify the centroids of various seeds, which specifically includes the following steps: The spliced ​​image is imported into the YOLOv8 recognition algorithm model to accurately identify the rice seeds in the seed hole and derive the pixel coordinates of the four vertices of each seed. These four vertices belong to the prediction box of the seed. The pixel coordinates of each vertex of the prediction box are obtained, A(y0, x0), B(y0, x1), C(y1, x0), D(y1, x1), and then the pixel coordinates of the seed centroid c (x m ,y m ), as shown in formula (6), 6. The sowing quality detection method based on image splicing algorithm and cluster analysis according to claim 1 is characterized in that: The step S4 specifically includes the following steps: S4.

1. Determine the centroid pixel coordinates of each seed hole and use the Bayesian Gaussian mixture model to determine the centroid coordinates of each seed hole (C x , C y ); S4.

2. Calculate the hole diameter and obtain the maximum distance d from all seeds in each hole to the centroid. max , to determine the hole diameter, as shown in formula (7), S4.

3. Calculate the hole distance. First determine the pixel value of the distance between the centers of the circumscribed circles of two adjacent seed holes. Then, in the camera coordinate system, combined with the camera height, calculate the distance between the centers of the circumscribed circles of the seeds of the two adjacent holes. As shown in (8), the actual distance S between the centroids of the two holes is obtained. Where Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the detection device camera to the seed hole, mm; S is the distance between the detection holes, mm.

7. The sowing quality detection method based on image splicing algorithm and cluster analysis according to claim 6 is characterized in that: Before calculating the hole diameter and hole distance in step S4, the intrinsic and extrinsic parameter matrices of the camera are obtained, and the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system. The conversion formula between the pixel coordinates of the identified seed and the camera coordinates is shown in formula (9). In the above formula, Represents the pixel coordinates in the image, where x and y are coordinates on the image plane, and 1 is a homogeneous coordinate used to simplify matrix operations; is the intrinsic parameter matrix of the camera, du and dv are the scale factors from pixel units to camera coordinate units, x0 and y0 are the coordinates of the principal point of the image (usually the center of the image); is the focal length matrix of the camera, where f is the focal length of the camera. This matrix transforms the points in the camera coordinate system into the world coordinate system; It is the three-dimensional coordinate in the camera coordinate system. By multiplying these matrices, the three-dimensional coordinate camera coordinate (X a , Y a , Z a ); is the external parameter matrix, where f u and f v are the components of the focal length in the u and v directions, x0 and y0 are the coordinates of the principal point; M represents the calibrated camera intrinsic parameter matrix.

8. The method for detecting rice sowing quality based on image stitching algorithm and cluster analysis according to claim 6, characterized in that: Step S4.1 includes: (1) Initialization processing Assume that the data is generated by a mixture of K Gaussian distributions, and the mean μ of each Gaussian distribution is k Corresponding to the centroid of a cluster; Including, initializing the parameters of the Gaussian distribution: the mean μ of each Gaussian distribution k , covariance matrix Σk and weight π k Initialize the data by randomly selecting data points as the mean, initialize the covariance matrix to the unit matrix or the covariance matrix of the data, and the weight π k Initialized to uniform distribution; (2) Expectation step: For each seed centroid position x i , calculate the posterior probability r that it belongs to the kth Gaussian distribution ik , the formula is: p(x|θ k ) is the probability density function of the kth Gaussian distribution, x is the seed centroid position vector, θ k =(μ k ,Σk,π k ) is the parameter of the k-th Gaussian distribution, d is the dimension of the data, is the probability density function of the entire Gaussian mixture model; (3) Maximization step: Update the model parameters based on the posterior probability calculated according to the expectation to maximize the log-likelihood function of the observed data. Update the mixing weight π k : Where N is the total number of seed centroids; Update mean μ k : Among them, x i represents a data point; Update the covariance matrix ∑k: Among them, (x i -μ k )(x i -μ k ) T is the data point x i and mean μ k The outer product of the deviations; (5) Iteration By repeatedly performing the expectation step and the maximization step until the parameters converge; (6) Determine the cluster centroid Calculate the centroid: After convergence, the centroid of the seed hole is calculated using the mean μ of each Gaussian distribution k According to the weight π k The weighted sum is obtained, that is,

Images