A sowing quality detection method based on image stitching algorithm and clustering analysis

Abstract

The present application belongs to the field of intelligent agriculture and information-based agriculture technology, and particularly relates to a sowing quality detection method based on image stitching algorithm and clustering analysis, namely a method for measuring rice hole diameter and hole distance based on image stitching mode of "speed-frame rate" and Bayesian Gaussian mixture model. The single seed hole image is stitched using "speed-frame rate" to obtain a seed bed panoramic image, and then the Bayesian Gaussian mixture model is used to cluster the seed holes, the features of each clustered seed hole are extracted, the maximum circumscribed circle of the seed hole is obtained, and the hole diameter of each seed hole is calculated. After obtaining the circumscribed circle of each seed hole, the pixel distance of the hole distance is calculated using the centroids of the adjacent two seed holes. The actual hole diameter and hole distance are obtained according to the conversion between pixel coordinates and world coordinates. The present application solves the problem that the single frame image cannot measure the adjacent hole distance, and the image detection method is more efficient and accurate compared with the video stream.

Description

Technical Field

[0001] This invention belongs to the field of smart agriculture and information-based agriculture technology, specifically involving a sowing quality detection method based on image stitching algorithm and cluster analysis, namely, a method for measuring the diameter and spacing of rice planting holes based on an image stitching algorithm of "speed-frame rate" and a clustering algorithm of Bayesian Gaussian mixture model. Background Technology

[0002] Mechanized rice cultivation methods include three types: direct seeding, mechanical transplanting, and mechanical broadcasting (Wang Zaiman et al., 2015). Direct seeding eliminates the traditional seedling raising and transplanting steps, directly sowing germinated rice seeds in the field, reducing operational steps and saving costs (Li Zehua et al., 2018). Among them, precision direct seeding of rice, as an advanced representative of direct seeding technology, has excellent seed-saving performance and has been widely promoted and applied in many provinces and cities in China (Zhang Shun et al., 2023). For example, Zhang Guozhong et al. designed two seed-stirring teeth devices with the same shape but different installation positions on the seed suction plate, which improved the sowing accuracy of the pneumatic seed metering device and had a significant seed-saving effect (Zhang Guozhong et al., 2013). Zhai Jianbo et al. designed a pneumatic precision direct seeding device for hybrid rice, which solved the problem of the flow of hybrid rice in the seed metering device (Zhai Jianbo et al., 2016). Xing He et al. designed a multi-channel air suction chamber structure, which can adjust the number of air channels to adapt to different seeding requirements (Xing He et al., 2019). To address the limitation of pneumatic seed metering devices in adapting their suction holes to different rice seed sizes, Zhang Kaixing et al. designed a double-circular suction plate that adjusts the size of the suction holes according to seed size (Zhang Kaixing et al., 2019). Zang Ying et al., utilizing the principle of flow guidance, designed a flow-guiding suction plate within the seed metering device.

[0003] The precision of single-seed sowing in hybrid rice has been improved (Zang Ying et al., 2021). The aforementioned research on the improvement and optimization of precision direct seeding technology for rice has promoted the mechanization level of precision direct seeding for rice.

[0004] With the deepening research on seed metering technology, the demand for rapid evaluation of seed metering device performance is increasing. Currently, domestic and international methods for testing seed metering device performance mainly include: manual inspection, photoelectric inspection, piezoelectric inspection, capacitance inspection, and machine vision inspection (Chi Peng et al. 2025, Ding et al. 2021). Manual inspection relies primarily on subjective evaluation of sowing quality by the human eye, directly detecting seed metering device performance with the naked eye; it is only suitable for situations where the seed metering device rotation speed is low. The photoelectric effect detection method utilizes optical and electronic principles. By installing photoelectric sensors on both sides of the seed metering tube, the falling seeds are identified by the pulse signals emitted by the sensors during the seeding process. The photoelectric effect detection method has been widely researched and applied due to its advantages such as automation, fast response speed, and ability to continuously process large amounts of data. This method has extremely high accuracy for detecting missed sowing, but when double sowing occurs, such as two or more seeds falling simultaneously, the system cannot count the overlapping seeds. Compared to photoelectric detection, the piezoelectric effect detection method requires ensuring that the falling seeds collide with the sensor to obtain a sensing signal. It also struggles to accurately determine the number of seeds when multiple seeds simultaneously impact the sensor. High-speed video recording can clearly reflect the movement of seeds within the seed metering device and during their fall, allowing for analysis of the device's performance using manual or image processing techniques. The biggest advantage of this method is that the high-speed camera can continuously record the seed trajectory in a very short time. However, this method suffers from drawbacks such as the high cost of high-speed cameras, stringent environmental requirements, and long post-experiment processing times. Furthermore, it requires a high-speed camera and supplementary lighting, making it suitable only for indoor research.

[0005] The existing technology CN116452526A, a method for rice seed identification and counting based on image detection, involves evenly scattering rice seeds on a white grid paper, taking a top-down shot with the camera lens parallel to the grid paper, and then training a deep convolutional neural network model on the captured images. The acquired rice seed images are divided into multiple sub-images and fed into the trained classification model. The number of rice seeds corresponding to the classification result of each rice seed image in the original rice seed image is then counted to obtain the total number of rice seeds in the original image. However, this method cannot simulate the actual field planting environment, meaning it cannot be practically applied to real-time detection of rice seed sowing in real-world environments. Furthermore, it only counts the number of rice seeds and cannot detect the sowing quality in real-time under actual farmland conditions.

[0006] In recent years, with the development of deep learning and industrial camera technology, scholars at home and abroad have gradually applied machine vision to seed quality inspection. Compared to traditional methods, machine vision inspection has significant advantages in real-time performance, accuracy, automation level, and information acquisition, improving the performance and intelligence of seed metering devices. Currently, with the rapid development of machine vision technology, more and more seed inspection technologies are adopting this approach, and inspection technology is developing towards intelligence, improving the accuracy and efficiency of seed inspection and gradually replacing the traditional manual sampling inspection methods that are labor-intensive, tedious, and inefficient. Summary of the Invention

[0007] To detect rice sowing quality, particularly measuring the diameter and spacing of rice planting holes, this invention provides an image stitching algorithm based on "speed-frame rate" linkage. This algorithm stitches together images of rice planting holes to generate a panoramic view of the rice seedbed, reflecting the overall distribution of seeds within the detection area. Then, a Bayesian Gaussian Mixture Model (BGMM) clustering algorithm is used to model each rice seed hole. First, an optimized recognition algorithm accurately identifies the rice seeds in the planting holes. Then, the BGMM algorithm is used to perform cluster analysis on the planting holes to obtain the centroid and circumcircle of the holes. The diameter of the circumcircle is calculated to obtain the hole diameter, and the distance between the centroids of two adjacent planting holes is calculated to obtain the hole spacing. This allows for quantitative analysis of sowing quality.

[0008] The specific technical solution of this invention is as follows: a method for detecting sowing quality based on image stitching algorithm and cluster analysis, comprising the following steps:

[0009] S1. Obtain multiple original rice seed hole sample images from farmland;

[0010] S2. Based on all the original rice seed hole sample images, synthesize a sample image of the panoramic view of the seed bed;

[0011] S3. Use image processing models to process the panoramic image of the seedbed and identify the centroids of various seeds;

[0012] S4. Using cluster analysis, cluster the centroids of the seeds identified in step 3 to divide the seeds in the seedbed into holes and determine the centroid of each hole. At the same time, calculate the diameter of each hole and the distance between adjacent holes.

[0013] In step S1, acquiring multiple original rice seed hole sample images specifically includes the following steps:

[0014] S1.1: As the seed metering device of the seed planter moves forward, an industrial camera is used to photograph the farmland to obtain rice sample image data; S1.2: Construct an image database of rice seed holes.

[0015] Step S1.2, which involves constructing an image database of rice seed holes, specifically includes the following steps:

[0016] S1.2.1: Preprocessing of rice seed image data, removing irrelevant information from the image background, and removing blurry or damaged images;

[0017] S1.2.2: Use the Labelimg annotation tool to annotate the preprocessed rice seed hole images and generate corresponding annotated rice seed hole image sample images;

[0018] S1.2.3: Dataset partitioning: The labeled dataset is randomly divided into a training set and a test set in a 6:4 ratio;

[0019] S1.2.4: Data augmentation processing. Online data augmentation techniques are used during training to transform the images in the training set in real time. Specifically, image processing techniques include scaling, random cropping, random rotation, and random horizontal flipping.

[0020] Step S2, which synthesizes a panoramic image of the rice seedbed based on all original rice seed hole sample images, specifically includes the following steps:

[0021] S2.1 Find the overlapping area between each frame of rice seed hole sample image and the next frame of rice seed hole sample image, and take the non-overlapping area as the area to be stitched.

[0022] S2.2. Cut off the overlapping areas of each rice seed hole sample image except the last frame of the rice seed hole sample image, obtain the area to be stitched, and stitch the area to be stitched to the starting frame of the next frame of the rice seed hole sample image according to the order of frame acquisition time. After stitching all the areas to be stitched, obtain the panoramic view of the seed bed.

[0023] Step S3 uses an image processing model to process the panoramic image of the seed bed and identify the centroids of various seeds. Specifically, it includes the following steps:

[0024] The stitched image is imported into the YOLOv8 recognition algorithm model to accurately identify rice seeds in the seed pit. The pixel coordinates of the four vertices of each seed are exported. These four vertices belong to the seed's prediction bounding box, and the pixel coordinates of each vertex of the prediction bounding box are obtained as A(y0, x0), B(y0, x1), C(y1, x0), and D(y1, x1). Then, the pixel coordinates (x, y0, x1) of the seed's centroid c are calculated. m y m As shown in equation (6),

[0025] Step S4 specifically includes the following steps:

[0026] S4.1 Determine the centroid pixel coordinates of each seed hole using a Bayesian Gaussian mixture model (C). x C y );

[0027] S4.2 Calculate the hole diameter to obtain the maximum distance d from all seeds in each hole to the centroid. max To determine the diameter of the acupoint, as shown in equation (7),

[0028]

[0029] S4.3 Calculate the distance between holes. First, determine the pixel value of the distance between the centers of the circumcircles of two adjacent holes. Then, in the camera coordinate system, combined with the camera height, calculate the distance between the centers of the circumcircles of the adjacent holes, as shown in (8), to obtain the actual distance S between the centers of the two holes.

[0030]

[0031] In the formula, Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the camera of the detection device to the planting hole, in mm; and S is the distance between the detection planting holes, in mm.

[0032] Before calculating the hole diameter and hole distance in step S4, the intrinsic and extrinsic parameter matrices of the camera are obtained, and the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system.

[0033] The formula for converting the pixel coordinates of the identified seed to the camera coordinates is shown in equation (9).

[0034]

[0035] In the above formula, This represents the pixel coordinates in the image, where x and y are coordinates on the image plane, and 1 represents homogeneous coordinates, used to simplify matrix operations;

[0036] It is the camera's intrinsic parameter matrix, where du and dv are scaling factors from pixel units to camera coordinate units, and x0 and y0 are the coordinates of the principal point of the image (usually the center of the image).

[0037] It is the camera's focal length matrix, where f is the camera's focal length. This matrix transforms points in the camera coordinate system to the world coordinate system.

[0038] These are the three-dimensional coordinates in the camera coordinate system. Through the multiplication of these matrices, the three-dimensional camera coordinates (X, Y, F) in the camera coordinate system can be obtained. a Y a Z a );

[0039] It is an extrinsic parameter matrix, where f u and f v These are the components of the focal length in the u and v directions, and x0 and y0 are the principal point coordinates; M represents the calibrated camera intrinsic parameter matrix.

[0040] Step S4.1 includes:

[0041] (1) Initialization process

[0042] Suppose the data is generated by a mixture of K Gaussian distributions, where the mean of each Gaussian distribution is μ. k This corresponds to the centroid of a cluster;

[0043] This includes initializing the parameters of the Gaussian distribution: the mean μ for each Gaussian distribution. k Covariance matrix Σk and weights π k Initialization is performed using randomly selected data points as the mean. The covariance matrix is ​​initialized to the identity matrix or the covariance matrix of the data, with weights π. k Initialize to a uniform distribution;

[0044] (2) Expected steps: For each seed centroid position x i Calculate the posterior probability r that it belongs to the k-th Gaussian distribution. ik The formula is:

[0045]

[0046] p(x|θ k ) is the probability density function of the k-th Gaussian distribution, x is the seed centroid position vector, and θ is the probability density function of the k-th Gaussian distribution. k =(μ k ,Σk,π k ) is the parameter of the k-th Gaussian distribution. d is the dimension of the data. Let be the probability density function of the entire Gaussian mixture model;

[0047] (3) Maximization step: Update the model parameters based on the expected posterior probability to maximize the log-likelihood function of the observed data.

[0048] Update the mixed weights π k : Where N is the total number of seed centroids;

[0049] Update mean μ k : Where, x i Represents data points;

[0050] Update the covariance matrix ∑k:

[0051] Where · represents the data point x i With mean μ k The outer product of the deviations;

[0052] (5) Iteration

[0053] By repeatedly performing the desired step and the maximization step, the parameters converge;

[0054] (6) Determine the centroid of the cluster

[0055] Centroid calculation: After convergence, the centroid of the planting hole is calculated using the mean μ of each Gaussian distribution. k According to the weight π k We get the weighted sum, that is

[0056] Compared to existing technologies, the advantages of this invention are as follows: an image stitching method based on "speed-frame rate" linkage solves the problem of the inability to precisely measure the distance between adjacent holes in a single frame image. Compared to video streams, image detection methods are more efficient and accurate. Based on the established rice seed detection model, a Bayesian Gaussian mixture model is used to divide the planting holes, and the geometric center of the rice seed in each hole is calculated. Finally, scale verification and bench test verification were carried out. The hole diameter and hole distance values ​​detected based on the Bayesian Gaussian mixture model were compared with manually measured values ​​to obtain the relative average error. The results of the relative average error show that the hole diameter and hole distance can be detected well under both laboratory and field conditions. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating the determination of the overlapping area between two adjacent frames of rice seed hole sample images in Example 1.

[0058] Figure 2 This describes the method for stitching two adjacent frames in Example 1;

[0059] Figure 3 This refers to the panoramic splicing method for the seedbed in Example 1;

[0060] Figure 4 This is a schematic diagram illustrating the acquisition of the center point coordinates of the seed prediction box using the YOLOv3 recognition algorithm model in Example 1.

[0061] Figure 5 This is a clustering result diagram of the Bayesian-Gaussian mixture model in Example 1;

[0062] Figure 6 This is a diagram showing the rice seed clustering results obtained by the BGMM clustering algorithm in Example 1.

[0063] Figure 7To take pictures of the chessboard pattern from different angles using a camera;

[0064] Figure 8 A schematic diagram for extracting feature points for chessboard calibration;

[0065] Figure 9 These are specific internal parameters of the industrial camera after calibration using the Zhang Zhengyou calibration method in Example 1;

[0066] Figure 10 The process of converting rice seed pixel coordinates to camera coordinates in Example 1.

[0067] Figure 11 A comparative analysis was conducted between the detection values ​​obtained from the clustering analysis model in Example 1 and the values ​​obtained from manual measurement.

[0068] Figure 12 The scale verification results are from Example 2;

[0069] Figure 13 This is a physical image of the tracked vehicle platform in Example 2;

[0070] Figure 14 The detection values ​​of the cluster analysis model in Example 2 are compared and analyzed with the values ​​obtained by manual measurement. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0072] Example 1

[0073] This embodiment is a detection method based on a pneumatic rice precision direct seeding machine detection system. The system includes an industrial camera, a host computer, a core processor, and a power supply. The pneumatic rice precision direct seeding machine has a camera bracket with an adjustable detection height based on a slide rail design to facilitate adjustment of different detection heights.

[0074] During direct seeding of rice, the field requires maintaining appropriate moisture, creating a complex working environment. Furthermore, rice seeds are smaller in size compared to crops like corn, thus placing higher demands on the performance of the detection system. The image acquisition device, acting as the "eyes" of the detection system, significantly impacts its accuracy. Industrial cameras, due to their high performance, stability, and reliability, effectively meet the requirements for extracting rice seed images during the sowing process, thereby contributing to the accuracy and stability of the detection system.

[0075] The seed metering device of the dual-cavity pneumatic precision rice seeder produces rice seeds with a relatively obvious hill-forming pattern, with an average of 3 to 5 seeds per hill. Cluster analysis can be used to analyze the sowing quality.

[0076] By acquiring multiple original rice planting hole sample images and constructing a rice planting hole image database, it is easier to manage the original rice planting hole sample images, thereby facilitating subsequent image processing and training, and obtaining a neural network planting hole recognition model with a high recognition rate. By processing all the original rice planting hole sample images in the rice planting hole image database, it is easier to obtain a high-quality target rice dataset, providing samples for subsequent deep learning methods, thereby facilitating subsequent deep learning training, effectively improving training quality, and thus improving the accuracy of the neural network rice ear recognition model in locating and recognizing the rice ears to be detected.

[0077] This embodiment mainly focuses on the quality inspection of rice seed sowing based on indicators such as the number of seeds per hole, hole diameter, hole spacing, and hole formation. The specific steps are as follows:

[0078] S1. Obtain multiple original rice seed hole sample images from farmland;

[0079] S2. Based on all the original rice seed hole sample images, synthesize a sample image of the panoramic view of the seed bed;

[0080] S3. Use the YOLOv3 deep learning image processing model to process the panoramic image of the seed bed and identify the centroids of various seeds;

[0081] S4. Using cluster analysis, cluster the centroids of the seeds identified in step 3 to divide the seeds in the seedbed into holes and determine the centroid of each hole. Classify the seed hole groups with different densities and calculate the hole diameter and the distance between adjacent seed holes in each hole.

[0082] In step S1, multiple original rice seed hole sample images are acquired, specifically including the following steps:

[0083] S1.1: As the seed metering device of the hill-planting machine moves forward, an industrial camera is used to photograph the farmland to obtain rice sample image data.

[0084] Preferably, in this step, the forward speed of the direct seeding machine is 0.2 m / s, the rotation speed of the seed metering shaft is 25 r / min, and the camera frame rate is 24.2.

[0085] Preferably, in this step, a CMOS industrial camera with a GigE Vision data cable interface is used, specifically the Hikvision MV-CS050-10GC.

[0086] S1.2: Construct an image database and a rice seed hole recognition model, specifically including the following steps:

[0087] S1.2.1: Preprocessing of rice seed image data, removing irrelevant information from the image background, and removing blurry or damaged images.

[0088] Given that the acquired image background carries a large amount of irrelevant information, which has a certain impact on the overall detection accuracy and processing speed, preprocessing is performed on the original dataset to remove unnecessary interference, thereby improving the accuracy and speed of model recognition and detection in the next step.

[0089] S1.2.2: Use the Labelimg annotation tool to annotate the preprocessed rice seed hole images, generating corresponding annotated rice seed hole sample images. The annotations include the location of the seed holes (boundary boxes), etc.

[0090] S1.2.3: Dataset partitioning. The labeled dataset is randomly divided into training and test sets in a 6:4 ratio. This ensures that the distribution of the training and test sets is representative, effectively enhancing the model's generalization performance and preventing information from the test set from being carried over into the training set.

[0091] S1.2.4: Data augmentation processing. During training, online data augmentation techniques are used to transform the images in the training set in real time. This specifically includes the following:

[0092] Scaling: Adjusts the size of the image to simulate images taken at different shooting distances.

[0093] Random cropping: Randomly cropping a portion of the image increases the model's ability to adapt to local features.

[0094] Random rotation: Randomly rotate the image by a certain angle to enhance the model's robustness to different shooting angles.

[0095] Random horizontal flip: Randomly flip the image horizontally to further enrich the dataset.

[0096] Therefore, in establishing the image database of rice seed holes, the first step is preprocessing. The purpose of preprocessing is to clean the data and remove irrelevant information, which is the foundation for subsequent annotation and model training. If annotation is performed before preprocessing, it may lead to lost or inaccurate annotation information. The second step is to annotate and segment the preprocessed images. Annotation precedes segmentation because it is performed on the original or preprocessed images; only after annotation is completed can the dataset be segmented. Segmenting the dataset ensures the independence of training and testing, avoiding data leakage. The third step is data augmentation. Data augmentation is performed during the training phase: it is applied to the training set to dynamically generate more varied images during training, enhancing the model's generalization ability. The test set does not require data augmentation to ensure the objectivity of the test results. These steps construct a rich dataset, preparing for the next step of improving the model's generalization ability.

[0097] In step S2 above, the obtained panoramic image of the seedbed is obtained based on an image stitching algorithm that considers the moving speed of the seeder and the frame rate of the camera, and includes the following specific steps:

[0098] S2.1 Find the overlapping area between each frame of rice seed hole sample image and the next frame of rice seed hole sample image, and take the non-overlapping area as the area to be stitched.

[0099] Taking the rice seed hole sample image obtained in step S1.1 as the object, this image accurately presents the overall distribution of seeds in the field. Multiple single-frame seedbed images are stitched together to create a panoramic view of the rice seeds, thereby visualizing the seed distribution throughout the detection area. The first step in stitching the panoramic view is to find the overlapping parts of the image stitching area. Analysis reveals that the camera's field of view changes with the movement of the seeder, and the seed holes captured at times t1 and t2 are different. Figure 1 As shown, there is an overlapping area between the two frames, which is represented by O1.

[0100] Set the seeder's forward speed to V1. The camera moves forward with the seeder, capturing images at a frame rate of f1. The size of a single frame is h1 × w1, where h1 is the height of the single frame and w1 is the width of the single frame. Figure 1 As shown, during the acquisition time interval t between two adjacent frames, the camera's field of view moves a distance l1 with the seeder, and the size of the non-overlapping area between the two adjacent frames is h1×l1. The time interval t between two adjacent images can be calculated from the acquisition frame rate of the image in Equation 1. The field of view movement distance l1 corresponding to each frame during image acquisition can be calculated from Equation 2. Therefore, the width of the overlapping area l2 can be calculated according to Equation 3:

[0101]

[0102] S2.2. Cut off the overlapping areas of each rice seed hole sample image except the last frame of the rice seed hole sample image, obtain the area to be stitched, and stitch the area to be stitched to the starting frame of the next frame of the rice seed hole sample image according to the order of frame acquisition time. After stitching all the areas to be stitched, obtain the panoramic view of the seed bed.

[0103] To better illustrate the image stitching process, we will first introduce the stitching of two images as an example, and then introduce the general panoramic stitching of seedbeds based on this. Figure 2 This is a method for stitching together two adjacent frames of images.

[0104] First, the overlapping areas in the previous frame are identified, and then these overlapping areas are truncated. The non-overlapping areas are retained and stitched to the starting frame of the next frame according to the order of frame acquisition time. Figure 3 (The middle image is the left side of the next frame). Finally, the heights of each frame are aligned to form the stitched image. The formulas for calculating the height h2 and width w2 of the stitched image are shown in Equation 4.

[0105]

[0106] like Figure 3 The diagram illustrates the method for panoramic stitching of the seedbed. First, overlapping areas in each frame are truncated, retaining only non-overlapping areas. Then, based on the direction of the seeder's movement and the chronological order of image acquisition, all non-overlapping areas are stitched together. The height of the panoramic seedbed image is H1, and the width is W1. The equations for H1 and W1 are shown in Equation 5.

[0107] m is the number of frames.

[0108] In this step, a panoramic image of the seedbed is obtained for each industrial camera, thus forming a sample image of the seedbed panoramic image, which is used for training and learning the image processing model in the next step.

[0109] In step S3 above, the panoramic image of the seedbed obtained in step S2 is imported into the image recognition model to obtain the centroid of each seed. In this step, a trained YOLOv8 image recognition model is used to extract features from the sample images of the target panoramic image of the seedbed. In the sample images of the entire panoramic image of the seedbed, there are many rice seeds, occupying a small area of ​​the entire image, and their distribution is messy and not concentrated. The YOLOv8 image recognition model is trained using multiple original rice seed hole sample images from step 1.

[0110] The stitched image is imported into a trained YOLOv8 image recognition model to accurately identify rice seeds in the seed holes, deriving the pixel coordinates of the four vertices of each seed. These four vertices belong to the seed's prediction bounding box. Using the direction of the live-streaming camera's movement as the y-axis and the direction perpendicular to the movement as the x-axis, the coordinates of each vertex of the seed prediction bounding box are set as A(y0, x0), B(y0, x1), C(y1, x0), and D(y1, x1), respectively. Figure 4 As shown. The pixel coordinates (x, y, y) of the seed centroid c can be calculated from equation (6). m y m ).

[0111]

[0112] After accurately identifying the centroids of each seed in a planting hole using YOLOv8, the next step is to perform cluster analysis on the centroids of the seeds in that planting hole. Therefore, in step S3, using a trained YOLOv8 to extract features from the dataset of the target rice seedbed panoramic image ensures that each rice seed in a planting hole can be effectively clustered, thereby accurately obtaining the cluster centers of the planting holes.

[0113] In step S4 above, the clustering analysis process only requires detecting the centroid pixel coordinates of two adjacent seeds for a single seed, then calculating the Euclidean distance between the centroids and converting it into the actual distance. However, for hill-sown rice seeds, each hill contains multiple seeds. In this case, it is necessary to first obtain the centroids of each seed in that hill, then use a Bayesian-Gaussian mixture model to perform clustering analysis to obtain the centroid of the planting hill, and then calculate the hill diameter and the distance between adjacent seeds.

[0114] Bayesian Gaussian Mixture Model (BGMM) is a clustering method based on Bayesian theory. It improves upon traditional Gaussian Mixture Models (GMMs) by introducing a Bayesian framework. BGMM can automatically determine the optimal number of clusters, and its clustering performance is as follows: Figure 5 As shown. In addition, BGMM automatically determines the optimal number of clusters through Bayesian inference, avoiding the limitation of KMeans clustering algorithm and AHC bottom-up hierarchical clustering method that require pre-specifying the number of clusters, while overcoming the parameter sensitivity problem of DBSCAN, a clustering algorithm that uses region density for clustering.

[0115] The method for determining the centroids of each cluster using a Bayesian Gaussian Mixture Model (BGMM) mainly includes the following steps:

[0116] (1) Initialization Processing Steps (Model Assumptions)

[0117] Choose the amount of Gaussian distributed data to determine the number of Gaussian distributions in the BGMM, i.e., the number of clusters. Assume the data is generated by a mixture of K Gaussian distributions, with each Gaussian distribution having a mean μ. k This corresponds to the centroid of a cluster.

[0118] This includes initializing the parameters of the Gaussian distribution: the mean μ for each Gaussian distribution. k Covariance matrix Σk and weights π k Perform initialization. Typically, randomly selected data points are used as the mean, and the covariance matrix is ​​initialized to the identity matrix or the data's covariance matrix, with weights π. k Initialize to a uniform distribution.

[0119] (2) Expected steps: For each seed centroid position x i Calculate the posterior probability r that it belongs to the k-th Gaussian distribution. ik The formula is:

[0120]

[0121] p(x|θ k ) is the probability density function of the k-th Gaussian distribution, x is the seed centroid position vector (x = (x, y) in the two-dimensional plane), and θ k =(μ k ,Σk,π k ) is the parameter of the k-th Gaussian distribution. d is the dimension of the data. Let be the probability density function of the entire Gaussian mixture model;

[0122] (3) Maximization step (M step): The goal of this step is to update the model parameters based on the expected posterior probability to maximize the log-likelihood function of the observed data.

[0123] Update the mixed weights π k : Where N is the total number of seed centroids.

[0124] Update mean μ k : Where, x i Represents a data point.

[0125] Update the covariance matrix ∑k:

[0126] Among them, (x i -μ k (x) i -μ k ) T It is data point x i With mean μ kThe outer product of the deviation.

[0127] (5) Iteration

[0128] The parameters of a BGMM can be optimized by repeatedly executing the expected step and the maximization step until the parameters converge (i.e., the change in parameters is less than a certain preset threshold or the maximum number of iterations is reached). In each iteration, the E-step provides the necessary statistical information to the M-step, which in turn uses this information to update the model parameters, thereby gradually improving the model's fit to the data.

[0129] (6) Determine the centroid of the cluster

[0130] Calculate the centroid: after the BGMM converges, the mean μ of each Gaussian distribution. i It can be regarded as the centroid of the corresponding cluster.

[0131] The centroid of the planting hole can be represented by the mean μ of various Gaussian distributions. k According to the weight π k We get the weighted sum, that is

[0132] Step S4 specifically includes the following steps:

[0133] S4.1 Obtain the centroid pixel coordinates of each seed hole: Use a Bayesian Gaussian mixture model to obtain the centroid coordinates of each seed hole (C x C y In this step, after completing the clustering operation, the clustering results are analyzed in depth, focusing on the rationality and stability of the clustering. After running the clustering algorithm, the model directly outputs the clustering effect diagram, which facilitates a clear understanding of the data distribution characteristics. To further verify the performance of the BGMM model, random samples are drawn from the collected sample image set, and the trained BGMM model is used to identify and detect the drawn samples. The rice seed clustering effect obtained by the BGMM clustering algorithm is shown below. Figure 6 As shown in the diagram, the visualization of clustering demonstrates that BGMM can accurately and stably divide each rice seed hole, laying the foundation for subsequent measurements of hole diameter and hole spacing.

[0134] S4.2 Calculate the hole diameter. For each seed in a hole, calculate the smallest circumcircle that contains all the seeds.

[0135] The pixel value of the hole diameter is calculated using equation (7).

[0136]

[0137] In the formula, (x i y i (C) represents the pixel coordinates of the i-th seed within the hole; x C y) represents the pixel coordinates of the centroid of the seed hole. n is the total number of seeds.

[0138] The diameter of a hole is the diameter of the circumcircle containing the most seeds in each hole, which can be calculated by taking the maximum distance d from all seeds to the centroid. max To determine.

[0139] S4.3 Calculate the distance between holes. First, determine the pixel value of the distance between the centers of the circumscribed circles of two adjacent holes. Then, in the camera coordinate system, calculate the distance between the centers of the circumscribed circles of two adjacent holes, as shown in Equation (8). This formula calculates the actual distance between the centers of the two holes.

[0140]

[0141] In the formula, Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the camera of the detection device to the planting hole, in mm; and S is the distance between the detection planting holes, in mm.

[0142] Step S4 also includes converting pixel coordinates into three-dimensional coordinates in the camera coordinate system using the camera's intrinsic and extrinsic parameter matrices. This step is typically performed before calculating the hole diameter and hole distance in step S4, because the position of the seeds in the camera coordinate system needs to be known in order to further calculate the actual distance between them.

[0143] In step S3, after obtaining the pixel coordinates of each seed in the rice seed hole and the pixel coordinates of the centroid of the seed hole, the camera's workspace is defined as the camera coordinate system, and the pixel coordinates (x, y) are transformed into camera coordinates (x, y). a Y a Z a ), and perform coordinate transformation.

[0144] When calibrating the camera, the discrepancy between the checkerboard pattern used and the actual height of the rice seeds being photographed can be resolved using a rigid transformation method. Camera calibration is crucial for vision systems, serving as the primary step in converting pixel distance into actual distance. After selecting the camera model, the Zhang Zhengyou calibration method was employed to obtain the camera's intrinsic parameter matrix. The selected checkerboard pattern was 16×13 pixels, containing 10mm black and white squares. More than 20 images of the checkerboard pattern under different poses were then taken with the camera. Figure 7 As shown.

[0145] After importing the captured images into Matlab, the built-in feature point extraction algorithm is used to detect the interior corners of the chessboard grid, thereby obtaining the pixel coordinates of the corresponding points, such as... Figure 8 As shown, this represents the extracted interior corner points of the chessboard grid. After calibrating the camera using Zhang Zhengyou's calibration method, the calculated calibration error is 0.14 pixels. Specific internal parameters of the camera are as follows: Figure 9As shown. The process of converting rice seed pixel coordinates to camera coordinates is as follows: Figure 10 As shown.

[0146] Depend on Figure 10 It can be seen that the conversion formula between the pixel coordinates of the identified seed and the camera coordinates is shown in equation (9).

[0147]

[0148] In the above formula, This represents the pixel coordinates in the image, where x and y are coordinates on the image plane, and 1 represents homogeneous coordinates, used to simplify matrix operations.

[0149] This is the camera's intrinsic parameter matrix, where du and dv are scaling factors from pixel units to camera coordinate units, and x0 and y0 are the coordinates of the principal point of the image (usually the center of the image).

[0150] This is the camera's focal length matrix, where f is the camera's focal length. This matrix transforms points in the camera coordinate system to the world coordinate system.

[0151] These are the three-dimensional coordinates in the camera coordinate system. Through the multiplication of these matrices, we can obtain the three-dimensional coordinates in the camera coordinate system: camera coordinates (X...). a Y a Z a ).

[0152] It is an extrinsic parameter matrix, where f u and f v These are the components of the focal length in the u and v directions, and x0 and y0 are the coordinates of the principal point.

[0153] In the formula: M represents the calibrated camera intrinsic parameter matrix, such as Figure 9 As shown in the image.

[0154] The pixel coordinates in the image are transformed into three-dimensional coordinates in the camera coordinate system using the camera's intrinsic and extrinsic parameter matrices. This transformation yields the camera coordinates corresponding to the pixel coordinates of each rice seed in the planting hole.

[0155] This conversion is crucial for precise positioning and manipulation in fields such as robot vision systems and agricultural automation. In this way, two-dimensional information in an image can be converted into information in three-dimensional space, thereby enabling precise object recognition and manipulation.

[0156] After the above transformation, the camera coordinates of the rice seeds in the image can be obtained, thereby calculating the actual diameter of the rice seed holes and the distance between adjacent seed holes. First, the circular coordinates of the circumcircle of each seed hole are determined, and then the Euclidean formula (as shown in Equation 10) is used to calculate the distance between the centers of the two circumcircles to obtain the actual seed hole distance.

[0157]

[0158] In the formula: (X1, Y1) and (X2, Y2) are the coordinates of the centers of two adjacent circumcircles.

[0159] S4.3 Verify the accuracy of the measurement of the diameter of the seed hole and the distance between adjacent seed holes in the cluster analysis model in the above steps.

[0160] To verify the practical application value of the above image stitching, recognition, clustering, and conversion process, 10 sample images of rice seed holes were selected from the collected images. The pixel coordinates of the rice seeds were converted into camera coordinates, and the actual hole diameter and hole spacing were calculated. The image stitching portion is shown below. Figure 2 and 3 As shown in the figure. Then, the model-based detection values ​​are compared and analyzed with the values ​​obtained by manual measurement, and the error rates are calculated respectively. The results are as follows. Figure 11 As shown.

[0161] To evaluate the recognition accuracy of the clustering model in step 4, the following steps are performed: Figure 11 The detection values ​​of acupoint diameter and acupoint distance are judged based on how close the relative mean error (MRE) value is to 0. MRE represents the relative average error between all model detection values ​​and human measurements, used to measure the average error of model detection values ​​relative to human measurements. The closer the MRE value is to 0, the higher the accuracy of the detection system, meaning the closer the model detection values ​​are to human measurements. The expression for the average relative error is shown in (4-10):

[0162]

[0163] In the formula: e represents the MRE value; n is the number of acupoints detected; y i The manually detected value for the i-th acupoint; Let be the model detection value of the i-th sample.

[0164] Calculations showed that the MRE value for hole diameter was 0.67 and the MRE value for hole spacing was 0.66, both approaching 0. This indicates that the measurement results based on the Bayesian Gaussian mixture model are relatively close to the manually measured values, thus confirming the feasibility of applying the cluster analysis method in step 4 to seed quality detection.

[0165] Example 2

[0166] To verify the effectiveness of the present invention, the clustering in step 4 of Example 1 was verified.

[0167] 1. Scale Verification

[0168] To further verify the feasibility of the clustering analysis model detection method in step 4 of the above embodiments, three sets of recognition verifications were conducted at different camera heights. Industrial cameras were installed on test benches at heights of 300mm, 400mm, and 500mm, respectively. Rice seeds at the 0 and 40 mark on the scale were then photographed, and the model was used for recognition and detection. The obtained model detection results are as follows: Figure 12 As shown.

[0169] Depend on Figure 12 It can be seen that the deviation between the detected distance and the actual value is within 3mm, which proves that the detection method is relatively accurate and has certain practical significance, and can be used to detect the quality of rice sowing.

[0170] 2. Bench test verification

[0171] Considering the minimal interference from external conditions on the detection system in a laboratory environment, and to more closely resemble a field testing environment, an intelligent tracked rice seeding device with single-row seeding was constructed. This device consists of a dual-chamber air-suction rice seed metering unit, a negative pressure unit, a motor, a motor speed control board, a camera, a computer, and a tracked trolley. The specific structure is as follows... Figure 13 As shown.

[0172] The motor speed of the seeding device can be adjusted using a STM32F103ZET6 microcontroller. The seeding negative pressure is fixed at -8 kPa, and the forward speed of the seeder can be adjusted by computer, with set speeds of 0.1 m / s, 0.2 m / s, and 0.3 m / s. The experimental field was located on the campus of Huazhong Agricultural University, and sowing was carried out at 3 PM with ample external sunlight. Before the experiment began, the motor speed was adjusted to 30 r / min, the tracked vehicle forward speed was adjusted to 0.1 m / s, and a 3-meter buffer zone was reserved. After the seeding effect of the seeder stabilized, video recording began. After recording, the images from each frame of the video were imported into the detection system, and the detection data was output. During the experiment, the number of seeds per hole, hole diameter, and hole spacing were measured manually. The experiment was divided into three groups, with 150 seeds measured in each group, and the average value was calculated.

[0173] The results obtained by the detection system were then compared with the results obtained by manual detection, as follows: Figure 14 As shown, the system's average error rate for detecting acupoint distance is 5.50%, and the average error for acupoint diameter is 13.25%.

[0174] right Figure 14The measured values ​​of hole diameter and hole spacing were also analyzed using the relative mean error (MRE) method. The final MRE value for hole diameter was 0.78, and the MRE value for hole spacing was 0.84, both of which were close to 0. The relative mean error results show that the measured values ​​based on the Bayesian Gaussian mixture model in step 4 of Example 1 are close to the manually measured values, proving that this method is feasible for sowing quality detection in a field environment.

[0175] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting sowing quality based on image stitching algorithm and cluster analysis, characterized in that, Includes the following steps, S1. Obtain multiple original rice seed hole sample images from farmland; S2. Based on all the original rice seed hole sample images, synthesize a sample image of the panoramic view of the seed bed; S3. Use image processing models to process the panoramic image of the seedbed and identify the centroids of various seeds; S4. Using cluster analysis, cluster the centroids of the seeds identified in step 3 to divide the seeds in the seedbed into holes and determine the centroid of each hole. At the same time, calculate the diameter of each hole and the distance between adjacent holes. Step S4 specifically includes the following steps: S4.1 Determine the centroid pixel coordinates of each seed hole using a Bayesian Gaussian mixture model (C). x C y ); S4.2 Calculate the hole diameter to obtain the maximum distance d from all seeds in each hole to the centroid. max To determine the diameter of the acupoint. ； S4.3 Calculate the distance between planting holes. First, determine the pixel value of the distance between the centers of the circumcircle of two adjacent planting holes. Then, in the camera coordinate system, combined with the camera height, calculate the distance between the centers of the circumcircle of the two adjacent planting holes to obtain the actual distance S between the centers of the two holes. ， In the formula, Δx and Δy are the horizontal and vertical distances between the centroids of two adjacent holes in the pixel coordinate system, Zc is the height from the camera of the detection device to the planting hole, mm; S is the distance between the detection planting holes, mm. Step S4.1 includes: (1) Initialization process Suppose the data is generated by a mixture of K Gaussian distributions, where the mean of each Gaussian distribution is μ. k This corresponds to the centroid of a cluster; This includes initializing the parameters of the Gaussian distribution: the mean μ for each Gaussian distribution. k Covariance matrix Σk and weights π k Initialization is performed using randomly selected data points as the mean. The covariance matrix is ​​initialized to the identity matrix or the covariance matrix of the data, with weights π. k Initialize to a uniform distribution; (2) Expected steps: For each seed centroid position x i Calculate the posterior probability r that it belongs to the k-th Gaussian distribution. ik The formula is: p(x|θ k ) is the probability density function of the k-th Gaussian distribution, x is the seed centroid position vector, and θ is the probability density function of the k-th Gaussian distribution. k =(μ) k ,Σk,π k ) is the parameter of the k-th Gaussian distribution. d is the dimension of the data. Let be the probability density function of the entire Gaussian mixture model; (3) Maximization step: Update the model parameters based on the expected posterior probability to maximize the log-likelihood function of the observed data. Update the mixed weights π k : , where N is the total number of seed centroids; Update mean μ k : , where x i Represents data points; Update the covariance matrix ∑k: ; in, It is data point x i With mean μ k The outer product of the deviations; (4) Iteration By repeatedly performing the desired step and the maximization step, the parameters converge; (5) Determine the centroid of the cluster Centroid calculation: After convergence, the centroid of the planting hole is calculated using the mean μ of each Gaussian distribution. k According to the weight π k We get the weighted sum, that is .

2. The sowing quality detection method based on image stitching algorithm and cluster analysis according to claim 1, characterized in that, In step S1, acquiring multiple original rice seed hole sample images specifically includes the following steps: S1.1: As the seed metering device of the seed planter moves forward, an industrial camera is used to photograph the farmland to obtain rice sample image data; S1.2: Construct an image database of rice seed holes.

3. The rice sowing quality detection method based on image stitching algorithm and cluster analysis according to claim 2, characterized in that, Step S1.2, which involves constructing an image database of rice seed holes, specifically includes the following steps: S1.2.1: Preprocessing of rice seed image data, removing irrelevant information from the image background, and removing blurry or damaged images; S1.2.2: Use the Labelimg annotation tool to annotate the preprocessed rice seed hole images and generate corresponding annotated rice seed hole image sample images; S1.2.3: Dataset partitioning: The labeled dataset is randomly divided into a training set and a test set in a 6:4 ratio; S1.2.4: Data augmentation processing. Online data augmentation techniques are used during training to transform the images in the training set in real time. Specifically, image processing techniques include scaling, random cropping, random rotation, and random horizontal flipping.

4. The sowing quality detection method based on image stitching algorithm and cluster analysis according to claim 1, characterized in that, Step S2, which synthesizes a panoramic image of the rice seedbed based on all original rice seed hole sample images, specifically includes the following steps: S2.1 Find the overlapping area between each frame of rice seed hole sample image and the next frame of rice seed hole sample image, and take the non-overlapping area as the area to be stitched. S2.

2. Cut off the overlapping areas of each rice seed hole sample image except the last frame of the rice seed hole sample image, obtain the area to be stitched, and stitch the area to be stitched to the starting frame of the next frame of the rice seed hole sample image according to the order of frame acquisition time. After stitching all the areas to be stitched, obtain the panoramic view of the seed bed.

5. The sowing quality detection method based on image stitching algorithm and cluster analysis according to claim 1, characterized in that, Step S3 uses an image processing model to process the panoramic image of the seed bed and identify the centroids of various seeds. Specifically, it includes the following steps: The stitched image is imported into the YOLOv8 recognition algorithm model to accurately identify rice seeds in the seed pit. The pixel coordinates of the four vertices of each seed are exported. These four vertices belong to the seed's prediction bounding box, and the pixel coordinates of each vertex of the prediction bounding box are obtained as A(y0, x0), B(y0, x1), C(y1, x0), and D(y1, x1). Then, the pixel coordinates of the seed's centroid c are calculated. m y m ), .

6. The sowing quality detection method based on image stitching algorithm and cluster analysis according to claim 1, characterized in that, Before calculating the hole diameter and hole distance in step S4, the intrinsic and extrinsic parameter matrices of the camera are obtained, and the pixel coordinates are converted into three-dimensional coordinates in the camera coordinate system. Formula for converting the pixel coordinates of the identified seed to the camera coordinates; ， In the above formula, This represents the pixel coordinates in the image, where x and y are coordinates on the image plane, and 1 represents homogeneous coordinates, used to simplify matrix operations; This is the camera's intrinsic parameter matrix, where du and dv are scaling factors from pixel units to camera coordinate units, and x0 and y0 are the coordinates of the principal point of the image. It is the camera's focal length matrix, where f is the camera's focal length. This matrix transforms points in the camera coordinate system to the world coordinate system. These are the three-dimensional coordinates in the camera coordinate system. Through the multiplication of these matrices, the three-dimensional camera coordinates (X, Y, F) in the camera coordinate system can be obtained. a Y a Z a ); It is an extrinsic parameter matrix, where f u and f v These are the components of the focal length in the u and v directions, and x0 and y0 are the principal point coordinates; M represents the calibrated camera intrinsic parameter matrix.

Images