Seed detection method, system and device based on multi-modal intelligent algorithm

Abstract

The invention provides a seed detection method, system and device based on a multi-modal intelligent algorithm. The method comprises the following steps: acquiring a raster image of a to-be-detected seed; rGB-D modal information of the raster image is collected; cutting an edge part in the raster image, and matching the cut image with the RGB-D modal information; performing target detection on the image by adopting a detection algorithm to determine a detection area of the seed in the image; establishing three-dimensional point cloud data according to the RGB-D modal information of the pixel points in the detection area; segmenting the three-dimensional point cloud data into point cloud data corresponding to each seed by adopting an adaptive density clustering algorithm; calculating volume data of each seed by using a point cloud convex hull algorithm according to the point cloud data; and clustering all volume data into real grain data and shriveled grain data by adopting a K-means clustering algorithm. According to the method disclosed by the invention, nondestructive seed test detection of the seeds is realized by adopting a multi-modal detection technology, and detection with higher precision is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of plant seed phenotype detection, and particularly relates to a seed detection method, system and device based on a multimodal intelligent algorithm. Background Art

[0002] Currently, China is committed to the research and development and optimization of intelligent agricultural equipment to reduce agricultural production costs, improve production efficiency, and promote the intelligent and automated development of seed quality detection.

[0003] Under this background, although some seed detection devices have been developed at home and abroad, these devices generally have problems such as single function, insufficient detection accuracy, and weak anti-interference ability. Especially under complex conditions such as incomplete threshing, seeds with branches and stalks or doped impurities, it is difficult to accurately identify shriveled grains and full grains. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a seed detection method, system and device based on a multimodal intelligent algorithm, which can accurately judge shriveled grains and full grains by combining multimodal recognition technology.

[0005] The present invention is implemented as follows: A seed detection method based on a multimodal intelligent algorithm includes the following steps: S1. Obtain a grid image of the seeds to be tested; S2. Collect the RGB-D modality information of the grid image; S3. Crop the edge part of the grid image and match the cropped image with the RGB-D modality information; S4. Use a detection algorithm to perform object detection on the image to determine the detection area of the seeds in the image; S5. Establish three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area; S6. Use an improved adaptive density clustering algorithm guided by machine learning to segment the three-dimensional point cloud data into point cloud data corresponding to each seed; S7. According to the point cloud data, use the point cloud convex hull algorithm to calculate the volume data of each seed; S8. Use the K-means clustering algorithm to cluster all volume data into full grain data and shriveled grain data.

[0006] Furthermore, the detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm.

[0007] Furthermore, the detection algorithm is trained through the following steps: S401. Annotate the image to determine the detection area; S402. Divide the labeled images into a training set and a test set; S403. Use the training set to train the detection algorithm to obtain a training result; S404. Use the test set to verify the training result, and after the training is completed, obtain the optimal weights of seeds of different varieties.

[0008] Further, the S6 includes the following sub-steps: S601. Perform preprocessing and feature extraction on the three-dimensional point cloud data to obtain the features of each point; S602. According to the features, use a density prediction model to predict the local density score of each point; S603. Adjust the key parameters of the DBSCAN clustering model according to the local density score; S604. Substitute the key parameters and the three-dimensional point cloud data into the DBSCAN clustering model for processing to obtain the point cloud data corresponding to each seed.

[0009] Further, it also includes S9. Calculate the seed setting rate according to the total number of seeds, the data of full seeds and the data of empty seeds; The S4 also includes determining the total number of seeds according to the number of detection regions.

[0010] The present invention also provides a seed detection system based on a multi-modal intelligent algorithm, including: An acquisition module, used to acquire a raster image of the seeds to be detected; A collection module, used to collect the RGB-D modal information of the raster image; A cropping module, used to crop the edge part of the raster image and match the cropped image with the RGB-D modal information; A detection module, used to perform object detection on the image by using a detection algorithm to determine the detection region of the seeds in the image; A three-dimensional point cloud data acquisition module, used to establish three-dimensional point cloud data according to the RGB-D modal information of the pixel points in the detection region; A clustering module, used to segment the three-dimensional point cloud data into the point cloud data corresponding to each seed by using an adaptive density clustering algorithm guided by machine learning; A volume calculation module, used to calculate the volume data of each seed by using a point cloud convex hull algorithm according to the point cloud data; A K-means clustering module, used to cluster all volume data into full seed data and empty seed data by using a K-means clustering algorithm.

[0011] Further, it also includes a seed setting rate calculation module, used to calculate the seed setting rate according to the total number of seeds, the full seed data and the empty seed data; The detection module also performs the step of determining the total number of seeds according to the number of the detection areas.

[0012] The present invention also provides a seed detection device based on a multimodal intelligent algorithm, including: A belt conveyor device, the belt conveyor device includes a conveyor belt, and a plurality of gratings for accommodating the seeds to be detected are formed on the part of the conveyor belt that bears the seeds to be detected; A hopper, the hopper is arranged above the discharging position of the belt conveyor device, and a feed door that realizes periodic opening and closing of the material port through an opening and closing device is arranged at the bottom thereof; A depth camera, the depth camera is arranged above the conveying path of the belt conveyor device; A weighing hopper, the weighing hopper is arranged below the end of the belt conveyor device to collect and detect the total weight of this batch of seeds; A control device, the control device executes the steps of the above method.

[0013] Further, the hopper is installed in an adjustable manner through an angle adjusting mechanism; The opening and closing device is installed at the bottom of the hopper, and it includes an opening and closing motor and an opening and closing cam; the opening and closing cam is installed below the feed door through an opening and closing cam shaft, and the motor shaft of the opening and closing motor is connected to the opening and closing cam shaft; the rim of the opening and closing cam forms a sliding contact with the bottom surface of the feed door.

[0014] Further, it further includes a flattening vibration device, and the flattening vibration device is located below the area between the discharging area and the detection area of the conveyor belt, so as to provide a vibration force to this area.

[0015] Further, an elastic guide plate is arranged on the front wall of the material port of the hopper, and after the feed door is closed, it can squeeze the lower edge of the elastic guide plate inward to deform it. After the feed door is opened, the lower edge of the elastic guide plate returns to its original state and forms a discharging channel with the feed door.

[0016] Further, a plurality of baffles are equidistantly distributed on the conveyor belt, and elastic side plates are arranged on both sides of the baffles; horizontal guide rods are arranged on both sides of the conveyor belt; during the process of the elastic side plates moving forward with the conveyor belt, they will overlap on the outside of the subsequent elastic side plates under the action of the horizontal guide rods, so as to form a grating for accommodating the seeds to be detected between the front and rear two baffles and the left and right two elastic side plates.

[0017] The beneficial effects brought by the present invention are: 1. The method in the present invention adopts multi-modal detection technology to achieve non-destructive seed measurement and inspection. Among them, using point cloud segmentation technology to detect the volume of seeds can avoid error problems caused by issues such as connected branches and impurities, thereby achieving higher-precision detection.

[0018] 2. The method in the present invention adopts the detection idea of first collecting multi-modal information and then calculating the volume. Compared with real-time detection methods, it can significantly reduce the computing power requirements, thereby reducing the hardware cost.

[0019] 3. The device in the present invention can better control the detection progress and frequency through the grid area on the conveyor belt, and realizes the accommodation of seeds to be measured through the grid with an improved structure, facilitating the acquisition of images with unified boundaries.

[0020] 4. The device in the present invention can achieve small-batch feeding at a time by adopting the method of vibrating feeding, minimizing the accumulation of seeds to be measured in the grid as much as possible, thereby improving the detection efficiency; the feeding hopper is installed in an angle-adjustable manner, facilitating the adjustment of the quantity of feeding at a time.

[0021] 5. The device in the present invention can achieve the flattening of seeds to be measured in the grid by adopting a vibration flattening device, reducing the situation of mutual overlapping and pressing, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flowchart of the method in the present invention; Figure 2 is Figure 1 a schematic diagram of the detection area in the method shown; Figure 3 is Figure 1 a schematic diagram of the point cloud in the method shown; Figure 4 is a three-dimensional structural diagram of the device in the present invention (the control device is not shown); Figure 5 is Figure 4 a partial enlarged view of I in; Figure 6 is Figure 4 a side view of the structure of the device shown; Figure 7 is Figure 6 a sectional view of the structure in the A-A direction in; Figure 8 is Figure 4 a front view of the structure of the device shown; Figure 9 is Figure 8 a sectional view of the structure in the B-B direction in; Figure 10 is Figure 9 a partial enlarged view of II in; Figure 11 is Figure 9 the partial enlarged view of III in the figure; Figure 12 is Figure 11 the enlarged view corresponding to the closed state of the material door.

[0023] Reference numerals: 1. Belt conveyor; 1.1. Conveyor belt; 1.2. Grille; 1.3. Baffle; 1.4. Elastic side plate; 1.5. Horizontal guide rod; 2. Hopper; 2.1. Material door; 2.2. Elastic material guide plate; 3. Depth camera; 4. Weighing hopper; 5. Opening and closing device; 5.1. Opening and closing cam; 5.2. Opening and closing motor; 6. Material guide port; 7. Angle adjustment mechanism; 8. Flattening vibration device; 8.1. Vibration motor; 8.2. Vibration cam; 8.3. Vibration rod. Specific implementation manners

[0024] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and cannot be used to limit the protection scope of the present invention.

[0025] Taking rice seeds as an example, as Figure 1 shown, the present invention provides a seed detection method based on a multi-modal intelligent algorithm, including the following steps: S1. Obtain the grid image of the seeds to be detected. In order to improve the recognition accuracy, the background of the grid image should be as monochromatic as possible and have a large contrast with the seeds to be detected.

[0026] S2. Collect the RGB-D modal information of the grid image.

[0027] S3. Crop the edge part of the grid image and match the cropped image with the RGB-D modal information.

[0028] S4. Use a detection algorithm to perform object detection on the image to determine the detection area of the seeds in the image; determine the total number of seeds according to the number of detection areas. In this step, the detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm. As Figure 2 shown, when using the YOLOv8 algorithm, the annotation box of the seeds is automatically generated to determine the detection area. When using the YOLOv8-seg algorithm, the mask image of the seeds is automatically generated to determine the detection area. When the system computing power is high, the YOLOv8-seg algorithm can be selected to execute the detection step. When the system computing power is low, the YOLOv8 algorithm can be selected to execute the detection step. The detection algorithm used in this step can automatically mark the detection area after training, and the training method includes the following steps: S401. Annotate the image to determine the detection area.

[0029] S402. Divide the labeled images into a training set and a test set.

[0030] S403. Train the detection algorithm with the training set to obtain a training result.

[0031] S404. Use the test set to verify the training result. After training, obtain the optimal weights for seeds of different varieties. The optimal weights obtained using the YOLOv8 algorithm are named "variety name + od+.pt", and the optimal weights obtained using the YOLOv8-seg algorithm are named "variety name + seg+.pt". It should be noted that when using the YOLOv8-seg algorithm for detection in this step, the distance between the two farthest pixels in each seed segmentation mask can also be calculated as the grain length, and the length of the line connecting the two farthest points perpendicular to the grain length in the mask can be calculated as the grain width. This data can be saved as grain shape data for other uses.

[0032] S5. Establish 3D point cloud data based on the RGB-D modality information of the pixels in the detection area.

[0033] S6. Use an adaptive density clustering algorithm guided by machine learning to segment the 3D point cloud data into point cloud data corresponding to each seed. Specifically, this step also includes the following sub-steps: S601. Preprocess and extract features from the 3D point cloud data to obtain the features of each point. In this step, preprocess the input point cloud data to ensure the consistency and validity of the data format. Then, use the deep learning model PointNet to extract the feature vector of each point. Let the point cloud data be , and its feature representation is , where is the feature vector of point .

[0034] S602. According to the features, use a density prediction model to predict the local density score of each point. In this step, construct a machine learning model based on density prediction to predict the local density score of each point. Specifically, this model is a classifier, with the input being the feature vector , and the output being the local density score of this point, indicating the probability that this point belongs to a high-density area.

[0035] S603. Adjust the key parameters of the DBSCAN clustering model according to the local density score. In this step, dynamically adjust the key parameters of the DBSCAN model according to the predicted density score. Specifically, the adaptive setting of the neighborhood radius ϵ: for each point , calculate the adaptive neighborhood radius as follows: ; where, is the basic neighborhood radius and α is the adjustment parameter.

[0036] Dynamic adjustment of the minimum sample number minPts: Set the minimum sample number minPts according to the local density score to adapt to the data distribution in different density regions.

[0037] S604. Substitute the key parameters and the three-dimensional point cloud data into the DBSCAN clustering model for processing to obtain the point cloud data corresponding to each seed, as Figure 3 shown. In this step, for each point , determine whether it is a core point, a boundary point or a noise point according to its adaptive neighborhood radius and the minimum sample number minPts. The determination method is the prior art and will not be elaborated here.

[0038] S7. According to the point cloud data, use the point cloud convex hull algorithm to calculate the volume data of each seed.

[0039] S8. Adopt the K-means clustering algorithm to cluster all the volume data into full grain data and shriveled grain data.

[0040] S9. Calculate the seed setting rate according to the total number of seeds, the full grain data and the shriveled grain data.

[0041] Based on the same inventive concept, the present invention also provides a seed detection system based on a multi-modal intelligent algorithm, including: An acquisition module, configured to acquire a raster image of the seed to be measured.

[0042] A collection module, configured to collect RGB-D modal information of the raster image.

[0043] A cropping module, configured to crop the edge part of the raster image and match the cropped image with the RGB-D modal information.

[0044] A detection module, configured to perform target detection on the image by using a detection algorithm to determine the detection area of the seed in the image. This module also performs the step of determining the total number of seeds according to the number of detection areas.

[0045] A three-dimensional point cloud data acquisition module, configured to establish three-dimensional point cloud data according to the RGB-D modal information of the pixel points in the detection area.

[0046] A clustering module, configured to segment the three-dimensional point cloud data into the point cloud data corresponding to each seed by using an adaptive density clustering algorithm guided by machine learning.

[0047] A volume calculation module, configured to calculate the volume data of each seed according to the point cloud data by using the point cloud convex hull algorithm.

[0048] A K-means clustering module, which is used to cluster all volume data into plump grain data and shriveled grain data by using the K-means clustering algorithm.

[0049] A seed setting rate calculation module, which is used to calculate the seed setting rate according to the total number of seeds, plump grain data and shriveled grain data.

[0050] As Figures 4 to 12 shown, based on the same inventive concept, the present invention also provides a seed detection device based on a multi-modal intelligent algorithm, which is applied to the detection method of the above S1 to S9, and includes a belt conveyor 1, a feed hopper 2, a depth camera 3 and a weighing hopper 4.

[0051] The belt conveyor 1 includes a conveyor belt 1.1, and the bracket of the belt conveyor 1 serves as the bracket of the entire device to install relevant components. A plurality of gratings 1.2 for accommodating the seeds to be measured are formed on the part of the conveyor belt 1.1 that bears the seeds to be measured. As Figure 4 and 5As shown, the purpose of setting the grille 1.2 is to facilitate the depth camera 3 to obtain a complete image of the seeds, so as to form a sequential detection image. That is to say, the seeds to be tested are housed in each grille 1.2, and the grille 1.2 serves as a boundary, so that the depth camera 3 can obtain a complete image including each grille 1.2 and the seeds to be tested housed therein. The grille 1.2 of the conveyor belt 1.1 in the prior art is composed of two side skirts and two front and rear baffles 1.3, and there are gaps between the side skirts and the baffles 1.3. Therefore, this grille 1.2 belongs to an open grille 1.2. When using this grille 1.2 structure to house the seeds to be tested, the seeds to be tested will fall into the gaps, so that a complete image of the seeds cannot be obtained. To overcome the above defects, the present invention has improved the structure of the grille 1.2: a number of baffles 1.3 are evenly distributed on the conveyor belt 1.1. The baffles 1.3 and the conveyor belt 1.1 are both made of rubber and are fixed by cementing. Elastic side plates 1.4 are provided on both sides of the baffle 1.3. The elastic side plates 1.4 are plastic thin plates or metal thin plates, so as to have the ability of deformation and rebound. The elastic side plates 1.4 are "L"-shaped structures, and one side structure thereof is fixed to both sides of the baffle 1.3 by cementing, and the other side structure extends backward. An included angle of 92-95° is formed between the two side structures of the elastic side plate 1.4. Horizontal guide rods 1.5 are provided on the brackets on both sides of the conveyor belt 1.1. When the elastic side plate 1.4 is located below or at the front and rear ends, it remains in an outwardly open state without being affected by the horizontal guide rod 1.5. When the elastic side plate 1.4 is located above and moves to the position of the horizontal guide rod 1.5, due to the extrusion of the horizontal guide rod 1.5, one side structure of the elastic side plate 1.4 will be deformed inward, and will overlap on the outside of one side structure of the elastic side plate 1.4 behind, so as to cooperate with the two front and rear baffles 1.3 to form a closed grille 1.2 to accommodate the seeds to be tested. In this way, after the grille 1.2 moves to the blanking position along with the conveyor belt 1.1, the seeds to be tested will fall into the grille 1.2, so that the depth camera 3 in the subsequent process can obtain a complete image. When the grille 1.2 moves to the end, the seeds fall into the weighing hopper 4, and one side structure of the elastic side plate 1.4 returns to its original state due to the loss of the extrusion of the horizontal guide rod 1.5.

[0052] Obviously, the grille 1.2 should be formed before the conveyor belt 1.1 reaches the landing point of the seeds to be tested, so as to prevent the seeds from appearing in other positions and affecting the detection. For the convenience of subsequent identification, the surface colors of the conveyor belt 1.1, the baffle 1.3 and the elastic side plate 1.4 should be the same or similar, and have a large contrast with the color of the seeds.

[0053] The blanking hopper 2 is installed on the bracket of the belt conveyor device 1 through a bracket and is located above the conveyor belt 1.1. A feed door 2.1 for periodically opening and closing the feed port is provided at the bottom of the blanking hopper 2 through an opening and closing device 5.

[0054] In order to reduce the situation of overlapping when the seeds to be tested fall into the grille 1.2, the hopper 2 uses vibratory feeding to make the seeds to be tested more dispersed. As Figure 4 、 Figure 11 and Figure 12 shown, the opening and closing device 5 is installed at the bottom of the hopper 2, which includes an opening and closing motor 5.2 and an opening and closing cam 5.1. The opening and closing cam 5.1 is installed below the door 2.1 through the opening and closing cam 5.1, and the motor shaft of the opening and closing motor 5.2 is connected to the shaft of the opening and closing cam 5.1. The rim of the opening and closing cam 5.1 forms a sliding contact with the bottom surface of the door 2.1. As Figure 9 shown, when the opening and closing motor 5.2 drives the opening and closing cam 5.1 to rotate until its small-diameter surface contacts the bottom surface of the door 2.1, the door 2.1 is in the open state, and at this time the seeds to be tested will fall into the grille 1.2 through the gap. When the opening and closing cam 5.1 rotates until its large-diameter surface contacts the bottom surface of the door 2.1, the door 2.1 is in the closed state, and at this time the seeds to be tested cannot fall. During a feeding cycle, the door 2.1 opens and closes multiple times to produce the effect of vibratory feeding, so that the seeds to be tested are dispersed in the grille 1.2.

[0055] In order to further eliminate the situation of seed overlapping, as Figure 9 and Figure 10 shown, the device also includes a flattening and vibrating device 8, which is located below the area between the feeding area and the detection area of the conveyor belt 1.1, so as to provide a vibration force to this area. The flattening and vibrating device 8 includes a vibration motor 8.1, a vibration cam 8.2 and a vibration rod 8.3. The vibration cam 8.2 is directly driven by the vibration motor 8.1 to rotate, and the vibration rod 8.3 is vertically slidably installed, and its lower end is in sliding contact with the outer edge surface of the vibration cam 8.2. The high-speed rotation of the vibration cam 8.2 drives the vibration rod 8.3 to quickly strike the conveyor belt 1.1, so as to make it generate high-frequency and low-amplitude vibrations. In this way, after the conveyor belt 1.1 is vibrated, the seeds to be tested tend to be flattened, avoiding aggregation and overlapping. And the low amplitude avoids large gaps between the elastic side plate 1.4 and the surface of the conveyor belt 1.1, resulting in seed leakage. The magnitude of the vibration force can be adaptively adjusted according to the seed quality and distribution, which belongs to conventional technology and will not be elaborated here.

[0056] As Figure 11 and Figure 12 shown, since the door 2.1 may get stuck with seeds and cannot be normally closed when it is closed, for this reason, an elastic guide plate 2.2 is provided on the front wall of the hopper 2 at the material outlet. After the door 2.1 is closed, it can squeeze the lower edge of the elastic guide plate 2.2 to deform it. After the door 2.1 is opened, the lower edge of the elastic guide plate 2.2 returns to its original state and forms a discharge channel with the door 2.1. By using the deformation ability of the elastic guide plate 2.2, when the door 2.1 is closed, it can squeeze the elastic guide plate 2.2 to deform inward, thus avoiding seed jamming.

[0057] To facilitate the discharge of all seeds, the hopper 2 is installed obliquely and can be adjusted through the angle adjustment mechanism 7. When the inclination angle changes, the number of seeds discharged each time will also change accordingly. The angle adjustment mechanism 7 uses an electric push rod.

[0058] The depth camera 3 is installed on the bracket of the belt conveyor 1 through a bracket and is located above the conveying path. After the grille 1.2 moves directly below the depth camera 3, the depth camera 3 captures images of the grille 1.2 and the seeds inside and transmits them to the control device. Since there are multiple grilles 1.2 formed on the conveyor belt 1.1, the depth camera 3 sequentially obtains images of each grille 1.2. When the system detects each image, if no seeds are detected in multiple consecutive images, a fault warning is issued. If the warning is ignored, it is considered that all the seeds in this batch have been detected.

[0059] The weighing hopper 4 is arranged below the end of the belt conveyor 1 to collect and detect the total weight of the seeds in this batch. The bottom of the weighing hopper 4 can be automatically opened after all the seeds in this batch have been detected to discharge the seeds. Since its structural part has not been improved, any weighing hopper 4 in the prior art can be used, so it will not be elaborated here. After all the seeds in this batch have been detected, the weighing hopper 4 will measure the total weight of the seeds in this batch, and the system will then calculate the thousand-grain weight of the seeds based on the number of seeds. A material guide port 6 is also arranged between the weighing hopper 4 and the end of the conveyor belt 1.1 to facilitate the seeds to slide into the weighing hopper 4.

[0060] Since the system detects plump seeds and shriveled seeds, in subsequent processes, the plump seeds can also be screened out by a winnowing device for later use.

[0061] It should be noted that the electrical control method of this device is completed by the control device. The above intelligent seed testing system is deployed in the control device. After the system runs, it executes steps S1 to S9. The rest of the electrical control principles and processes belong to the prior art, and the present invention does not involve improvements in this part. When this device is running, it executes steps S1 to S9 to complete the seed detection process for one batch. Specifically as follows: The seeds to be tested are imported into the hopper 2, and after selecting the seed type, the device is started. As the conveyor belt 1.1 moves, the grilles 1.2 are sequentially formed on the upper surface of the conveyor belt 1.1. At the same time, as the material gate 2.1 is repeatedly opened, the seeds gradually fall into each grille 1.2. When the seeds in the grille 1.2 come under the depth camera 3 along with the conveyor belt 1.1, the depth camera 3 captures the seed images and sends them to the control device. The control device executes the detection process to complete the detection of the volume of the seeds in each grille 1.2 and finally identifies plump seeds, shriveled seeds and their quantities.

[0062] When the system continuously detects that multiple grilles 1.2 have no seeds, the system issues a fault warning. The staff determines whether there is a fault at this time based on the warning. If not, it is considered that all seed detections are completed. At this time, the system calculates the thousand-seed weight based on the weight and the number of seeds detected by the weighing hopper 4. Thus, the entire detection process ends.

[0063] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A seed detection method based on a multimodal intelligent algorithm, characterized in that: The steps include: S1, obtaining a grid image of the seeds to be tested; S2, collecting RGB-D modality information of the raster image; S3, cropping the edge portion of the raster image, and matching the cropped image with the RGB-D modality information; S4, using a detection algorithm to perform target detection on the image to determine a detection area of ​​the seed in the image; S5, establishing three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area; S6. Using an adaptive density clustering algorithm guided by machine learning to segment the three-dimensional point cloud data into point cloud data corresponding to each seed; S7. Calculate the volume data of each seed using a point cloud convex hull algorithm according to the point cloud data; S8. Use K-means clustering algorithm to cluster all volume data into solid particle data and deflated particle data.

2. The seed detection method based on a multimodal intelligent algorithm according to claim 1, characterized in that: The detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm.

3. The seed detection method based on multimodal intelligent algorithm according to claim 2 is characterized in that: The detection algorithm is trained by the following steps: S401, marking the image to determine a detection area; S402, dividing the labeled images into a training set and a test set; S403, training the detection algorithm with the training set to obtain a training result; S404: Verify the training result using the test set, and obtain the optimal weights of seeds of different varieties after the training is completed.

4. The seed detection method based on a multimodal intelligent algorithm according to claim 1, characterized in that: The S6 comprises the following sub-steps: S601, preprocessing and feature extraction processing are performed on the three-dimensional point cloud data to obtain the features of each point; S602: predicting the local density score of each point using a density prediction model according to the features; S603, adjusting key parameters of the DBSCAN clustering model according to the local density score; S604: Substitute the key parameters and three-dimensional point cloud data into the DBSCAN clustering model to obtain point cloud data corresponding to each seed.

5. The seed detection method based on a multimodal intelligent algorithm according to claim 1, characterized in that: It also includes S9, calculating the seed setting rate based on the total number of seeds, the data of filled seeds and the data of shrunken seeds; The step S4 further includes determining the total number of seeds according to the number of the detection areas.

6. A seed detection system based on a multimodal intelligent algorithm, characterized in that: include: An acquisition module is used to acquire a raster image of the seed to be tested; An acquisition module, used for acquiring RGB-D modality information of the raster image; A cropping module, used for cropping edge portions of the raster image and matching the cropped image with the RGB-D modality information; A detection module, used to perform target detection on the image using a detection algorithm to determine a detection area of ​​the seed in the image; A three-dimensional point cloud data acquisition module, used to establish three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area; A clustering module, used for segmenting the three-dimensional point cloud data into point cloud data corresponding to each seed by using an adaptive density clustering algorithm guided by machine learning; A volume calculation module, used to calculate the volume data of each seed using a point cloud convex hull algorithm according to the point cloud data; The K-means clustering module is used to cluster all volume data into solid particle data and deflated particle data using the K-means clustering algorithm.

7. A seed detection system based on a multimodal intelligent algorithm according to claim 6, characterized in that: It also includes a fruiting rate calculation module, which is used to calculate the fruiting rate based on the total number of seeds, the data of filled grains and the data of shrunken grains; The detection module further performs a step of determining a total number of seeds according to the number of the detection areas.

8. A seed detection device based on a multimodal intelligent algorithm, characterized in that: include: A belt conveyor device (1), the belt conveyor device (1) comprising a conveyor belt (1.1), a portion of the conveyor belt (1.1) carrying the seeds to be tested being formed with a plurality of grids (1.2) for accommodating the seeds to be tested; A material discharge hopper (2), the material discharge hopper (2) being arranged above the material discharge position of the belt conveyor device (1), and having a material gate (2.1) at its bottom for realizing periodic opening and closing of a material opening through an opening and closing device (5); A depth camera (3), the depth camera (3) being arranged above the conveying path of the belt conveyor device (1); A weighing hopper (4), the weighing hopper (4) being arranged below the end of the belt conveyor (1) to collect and detect the total weight of the batch of seeds; A control device, wherein the control device executes the steps of a seed detection method based on a multimodal intelligent algorithm as described in claim 1.

9. A seed detection device based on a multimodal intelligent algorithm according to claim 8, characterized in that: The lower hopper (2) is adjustable via an angle adjustment mechanism (7); The opening and closing device (5) is installed at the bottom of the lower hopper (2), and comprises an opening and closing motor (5.2) and an opening and closing cam (5.1); the opening and closing cam (5.1) is installed below the material door (2.1) through the opening and closing cam (5.1), and the motor shaft of the opening and closing motor (5.2) is connected to the shaft of the opening and closing cam (5.1); the wheel rim of the opening and closing cam (5.1) forms a sliding contact with the bottom surface of the material door (2.1); An elastic material guide plate (2.2) is provided on the front wall of the material opening of the lower hopper (2); when the material door (2.1) is closed, the lower edge of the elastic material guide plate (2.2) can be pressed inwards to deform the lower edge; when the material door (2.1) is opened, the lower edge of the elastic material guide plate (2.2) returns to its original shape and forms a material discharge channel between the material door (2.1).

10. A seed detection device based on a multimodal intelligent algorithm according to claim 8, characterized in that: It also comprises a flattening vibration device (8), wherein the flattening vibration device (8) is located below the area between the unloading area and the detection area of ​​the conveyor belt (1.1), thereby providing a vibration force to the area.

11. The seed detection device based on multimodal intelligent algorithm according to claim 8, characterized in that: A plurality of baffles (1.3) are evenly spaced on the conveyor belt (1.1), and elastic side plates (1.4) are arranged on both sides of the baffles (1.3); horizontal guide rods (1.5) are arranged on both sides of the conveyor belt (1.1); and the elastic side plates (1.4) overlap the outer sides of the elastic side plates (1.4) at the rear under the action of the horizontal guide rods (1.5) as the conveyor belt (1.1) moves forward, thereby forming a grid (1.2) for accommodating seeds to be tested between the front and rear baffles (1.3) and the left and right elastic side plates (1.4).

Images