A seed detection method, system and device based on a multimodal intelligent algorithm

Abstract

The present invention provides a seed detection method, system and device based on a multimodal intelligent algorithm. The method includes the following steps: obtaining a raster image of the seeds to be measured; collecting the RGB-D modal information of the raster image; cropping the edge part of the raster image and matching the cropped image with the RGB-D modal information; using a detection algorithm to perform object detection on the image to determine the detection area of the seeds in the image; establishing three-dimensional point cloud data according to the RGB-D modal information of the pixel points in the detection area; using an adaptive density clustering algorithm to segment the three-dimensional point cloud data into point cloud data corresponding to each seed; according to the point cloud data, using a point cloud convex hull algorithm to calculate the volume data of each seed; using a K-means clustering algorithm to cluster all the volume data into full grain data and shriveled grain data. The method in the present invention uses multimodal detection technology to realize non-destructive seed measurement and achieve higher-precision detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of plant seed phenotype detection, and particularly relates to a seed detection method, system and device based on a multimodal intelligent algorithm. Background Art

[0002] Currently, China is committed to the research and development and optimization of intelligent agricultural equipment to reduce agricultural production costs, improve production efficiency, and promote the intelligent and automated development of seed quality detection.

[0003] Under this background, although some seed detection devices have been developed at home and abroad, these devices generally have problems such as single function, insufficient detection accuracy, and weak anti-interference ability. Especially under complex conditions such as incomplete threshing, seeds with branches and stalks or doped impurities, it is difficult to accurately identify shriveled grains and full grains. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to provide a seed detection method, system and device based on a multimodal intelligent algorithm, which can accurately judge shriveled grains and full grains by combining multimodal recognition technology.

[0005] The present invention is implemented as follows: A seed detection method based on a multimodal intelligent algorithm includes the following steps:

[0006] S1. Obtain a raster image of the seeds to be tested;

[0007] S2. Collect RGB-D modality information of the raster image;

[0008] S3. Crop the edge part of the raster image and match the cropped image with the RGB-D modality information;

[0009] S4. Use a detection algorithm to perform target detection on the image to determine the detection area of the seeds in the image;

[0010] S5. Establish three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area;

[0011] S6. Use an improved adaptive density clustering algorithm guided by machine learning to divide the three-dimensional point cloud data into point cloud data corresponding to each seed;

[0012] S7. According to the point cloud data, use the point cloud convex hull algorithm to calculate the volume data of each seed;

[0013] S8. Use the K-means clustering algorithm to cluster all volume data into full grain data and shriveled grain data.

[0014] Further, the detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm.

[0015] Further, the detection algorithm is trained through the following steps:

[0016] S401. Annotate the image to determine the detection area;

[0017] S402. Divide the annotated image into a training set and a test set;

[0018] S403. Train the detection algorithm with the training set to obtain a training result;

[0019] S404. Use the test set to verify the training result, and obtain the optimal weights of seeds of different varieties after the training ends.

[0020] Further, the S6 includes the following sub-steps:

[0021] S601. Perform preprocessing and feature extraction on the three-dimensional point cloud data to obtain the features of each point;

[0022] S602. According to the features, use the density prediction model to predict the local density score of each point;

[0023] S603. Adjust the key parameters of the DBSCAN clustering model according to the local density score;

[0024] S604. Substitute the key parameters and the three-dimensional point cloud data into the DBSCAN clustering model for processing to obtain the point cloud data corresponding to each seed.

[0025] Further, it also includes S9. Calculate the seed setting rate according to the total number of seeds, the data of full seeds and the data of empty seeds;

[0026] The S4 also includes determining the total number of seeds according to the number of detection areas.

[0027] The present invention also provides a seed detection system based on a multi-modal intelligent algorithm, including:

[0028] An acquisition module, used to acquire the raster image of the seeds to be detected;

[0029] A collection module, used to collect the RGB-D modal information of the raster image;

[0030] A cropping module, used to crop the edge part of the raster image and match the cropped image with the RGB-D modal information;

[0031] A detection module, used to perform target detection on the image by using a detection algorithm to determine the detection area of the seeds in the image;

[0032] A three-dimensional point cloud data acquisition module, configured to establish three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area;

[0033] A clustering module, configured to segment the three-dimensional point cloud data into point cloud data corresponding to each seed by using an adaptive density clustering algorithm guided by machine learning;

[0034] A volume calculation module, configured to calculate the volume data of each seed by using a point cloud convex hull algorithm according to the point cloud data;

[0035] A K-means clustering module, configured to cluster all volume data into full grain data and shriveled grain data by using the K-means clustering algorithm.

[0036] Furthermore, it further includes a seed setting rate calculation module, configured to calculate the seed setting rate according to the total number of seeds, the full grain data and the shriveled grain data;

[0037] The detection module further performs the step of determining the total number of seeds according to the number of the detection areas.

[0038] The present invention further provides a seed detection device based on a multi-modal intelligent algorithm, including:

[0039] A belt conveyor device, the belt conveyor device includes a conveyor belt, and a plurality of gratings for accommodating the seeds to be detected are formed on the part of the conveyor belt that bears the seeds to be detected;

[0040] A feeding hopper, the feeding hopper is arranged above the feeding position of the belt conveyor device, and a feed door for periodically opening and closing the material outlet is arranged at the bottom thereof through an opening and closing device;

[0041] A depth camera, the depth camera is arranged above the conveying path of the belt conveyor device;

[0042] A weighing hopper, the weighing hopper is arranged below the end of the belt conveyor device to collect and detect the total weight of this batch of seeds;

[0043] A control device, the control device executes the steps of the above method.

[0044] Furthermore, the feeding hopper is adjustably installed through an angle adjusting mechanism;

[0045] The opening and closing device is installed at the bottom of the feeding hopper, and includes an opening and closing motor and an opening and closing cam; the opening and closing cam is installed below the feed door through an opening and closing cam shaft, and the motor shaft of the opening and closing motor is connected to the opening and closing cam shaft; the rim of the opening and closing cam forms a sliding contact with the bottom surface of the feed door.

[0046] Furthermore, it also includes a flattening vibration device, which is located below the area between the unloading area and the detection area of ​​the conveyor belt, thereby providing a vibration force to the area.

[0047] Furthermore, an elastic material guide plate is provided on the front wall of the material port of the lower hopper. When the material door is closed, the lower edge of the elastic material guide plate can be squeezed inward to deform it. When the material door is opened, the lower edge of the elastic material guide plate returns to its original state and forms a discharge channel between the material door.

[0048] Furthermore, a number of baffles are evenly distributed on the conveyor belt, and elastic side panels are arranged on both sides of the baffles; horizontal guide rods are arranged on both sides of the conveyor belt; the elastic side panels will overlap the outer sides of the elastic side panels at the rear under the action of the horizontal guide rods as the conveyor belt moves forward, thereby forming a grid for accommodating the seeds to be tested between the front and rear baffles and the left and right elastic side panels.

[0049] The beneficial effects brought by the present invention are:

[0050] 1. The method of the present invention adopts multimodal detection technology to realize non-destructive seed testing. The point cloud segmentation technology is used to detect the seed volume, which can avoid the error caused by the problems of connecting branches, impurities, etc., thereby achieving higher accuracy detection.

[0051] 2. The method of the present invention adopts the detection idea of ​​first collecting multimodal information and then calculating the volume. Compared with the real-time detection method, it can significantly reduce the computing power requirements, thereby reducing the hardware cost.

[0052] 3. The device of the present invention can better control the progress and frequency of detection through the grid area on the conveyor belt, and can accommodate the seeds to be detected through the improved structure of the grid, so as to facilitate the acquisition of images with uniform boundaries.

[0053] 4. The device of the present invention adopts a vibration feeding method to achieve a single small batch feeding, which can reduce the accumulation of seeds to be tested in the grid as much as possible, thereby improving the detection efficiency; the feeding hopper is installed in an angle-adjustable manner to facilitate the adjustment of the amount of single feeding.

[0054] 5. The device of the present invention adopts a vibrating paving device to achieve paving of the seeds to be tested in the grid, reducing the situation of overlapping each other, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of the method of the present invention;

[0056] Figure 2 for Figure 1 A schematic diagram of the detection area in the illustrated method;

[0057] Figure 3 Schematic diagram of the point cloud in the method shown; Figure 1

[0058] Figure 4 Stereoscopic structure diagram of the device in the present invention (the control device is not shown);

[0059] Figure 5 Schematic diagram; Figure 4 Partial enlarged view at I in;

[0060] Figure 6 Schematic diagram; Figure 4 Side view of the structure of the device shown;

[0061] Figure 7 Schematic diagram; Figure 6 Cross-sectional view of the structure in the A-A direction in;

[0062] Figure 8 Schematic diagram; Figure 4 Front view of the structure of the device shown;

[0063] Figure 9 Schematic diagram; Figure 8 Cross-sectional view of the structure in the B-B direction in;

[0064] Figure 10 Schematic diagram; Figure 9 Partial enlarged view at II in;

[0065] Figure 11 Schematic diagram; Figure 9 Partial enlarged view at III in;

[0066] Figure 12 Schematic diagram; Figure 11 Enlarged view corresponding to the closed state of the material gate.

[0067] Reference numerals:

[0068] 1. Belt conveyor device; 1.1. Conveyor belt; 1.2. Grille; 1.3. Baffle; 1.4. Elastic side plate; 1.5. Horizontal guide rod; 2. Hopper; 2.1. Material gate; 2.2. Elastic guide plate; 3. Depth camera; 4. Weighing hopper; 5. Opening and closing device; 5.1. Opening and closing cam; 5.2. Opening and closing motor; 6. Material guide port; 7. Angle adjustment mechanism; 8. Spreading vibration device; 8.1. Vibration motor; 8.2. Vibration cam; 8.3. Vibration rod. Detailed implementation manners

[0069] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0070] Taking rice seeds as an example, as Figure 1 ​As shown in the figure, the present invention provides a seed detection method based on a multi-modal intelligent algorithm, including the following steps:

[0071] S1. Obtain the raster image of the seed to be detected. To improve the recognition accuracy, the background of the raster image should be as monochromatic as possible and have a large contrast with the seed to be detected.

[0072] S2. Collect the RGB-D modal information of the raster image.

[0073] S3. Crop the edge part of the raster image and match the cropped image with the RGB-D modal information.

[0074] S4. Use a detection algorithm to perform object detection on the image to determine the detection area of the seed in the image; determine the total number of seeds according to the number of detection areas. In this step, the detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm. As Figure 2 shown, when using the YOLOv8 algorithm, the annotation box of the seed is automatically generated to determine the detection area. When using the YOLOv8-seg algorithm, the mask image of the seed is automatically generated to determine the detection area. When the system computing power is high, the YOLOv8-seg algorithm can be selected to execute the detection step. When the system computing power is low, the YOLOv8 algorithm can be selected to execute the detection step. The detection algorithm used in this step can automatically mark the detection area after training. The training method includes the following steps:

[0075] S401. Annotate the image to determine the detection area.

[0076] S402. Divide the annotated image into a training set and a test set.

[0077] S403. Train the detection algorithm with the training set to obtain the training result.

[0078] S404. Use the test set to verify the training result. After training, the best weights of different varieties of seeds are obtained. The best weights obtained by using the YOLOv8 algorithm are named "variety name + od +.pt", and the best weights obtained by using the YOLOv8-seg algorithm are named "variety name + seg +.pt". It should be noted that when using the YOLOv8-seg algorithm for detection in this step, the distance between the two farthest pixels in the segmentation mask of each seed can also be calculated as the grain length, and the length of the line connecting the two farthest points perpendicular to the grain length in the mask can be calculated as the grain width. This data can be saved as grain shape data for other uses.

[0079] S5. Establish three-dimensional point cloud data according to the RGB-D modal information of the pixel points in the detection area.

[0080] S6. Use an adaptive density clustering algorithm guided by machine learning to segment the three-dimensional point cloud data into the point cloud data corresponding to each seed. Specifically, this step further includes the following sub-steps:

[0081] S601. Preprocess and perform feature extraction on the three-dimensional point cloud data to obtain the features of each point. In this step, preprocess the input point cloud data to ensure the consistency and validity of the data format. Then, use the deep learning model PointNet to extract the feature vector of each point. Let the point cloud data be , and its feature representation be , where is the feature vector of point .

[0082] S602. According to the features, use a density prediction model to predict the local density score of each point. In this step, construct a machine learning model based on density prediction to predict the local density score of each point. Specifically, this model is a classifier, with the input being the feature vector , and the output being the local density score of this point, representing the probability that this point belongs to a high-density area.

[0083] S603. Adjust the key parameters of the DBSCAN clustering model according to the local density score. In this step, dynamically adjust the key parameters of the DBSCAN model according to the predicted density score. Specifically, the adaptive setting of the neighborhood radius ϵ: For each point , calculate the adaptive neighborhood radius as follows:

[0084] ; where is the basic neighborhood radius, and α is the adjustment parameter.

[0085] The dynamic adjustment of the minimum sample number minPts: Set the minimum sample number minPts according to the local density score to adapt to the data distribution in different density regions.

[0086] S604. Substitute the key parameters and the three-dimensional point cloud data into the DBSCAN clustering model for processing to obtain the point cloud data corresponding to each seed, as shown in Figure 3 . In this step, for each point , determine whether it is a core point, a boundary point, or a noise point according to its adaptive neighborhood radius and the minimum sample number minPts. The determination method is the prior art and will not be elaborated here.

[0087] S7. According to the point cloud data, use the point cloud convex hull algorithm to calculate the volume data of each seed.

[0088] S8. Use the K-means clustering algorithm to cluster all volume data into full grain data and shriveled grain data.

[0089] S9. Calculate the seed setting rate based on the total number of seeds, full grain data, and shriveled grain data.

[0090] Based on the same inventive concept, the present invention also provides a seed detection system based on a multimodal intelligent algorithm, including:

[0091] An acquisition module for acquiring a raster image of the seeds to be measured.

[0092] An acquisition module for acquiring RGB-D modality information of the raster image.

[0093] A cropping module for cropping the edge part of the raster image and matching the cropped image with the RGB-D modality information.

[0094] A detection module for performing object detection on the image using a detection algorithm to determine the detection area of the seeds within the image. This module also performs the step of determining the total number of seeds based on the number of detection areas.

[0095] A three-dimensional point cloud data acquisition module for establishing three-dimensional point cloud data based on the RGB-D modality information of the pixel points within the detection area.

[0096] A clustering module for using an adaptive density clustering algorithm guided by machine learning to segment the three-dimensional point cloud data into point cloud data corresponding to each seed.

[0097] A volume calculation module for calculating the volume data of each seed using the point cloud convex hull algorithm based on the point cloud data.

[0098] A K-means clustering module for using the K-means clustering algorithm to cluster all volume data into full grain data and shriveled grain data.

[0099] A seed setting rate calculation module for calculating the seed setting rate based on the total number of seeds, full grain data, and shriveled grain data.

[0100] As Figures 4 to 12 shown, based on the same inventive concept, the present invention also provides a seed detection device based on a multimodal intelligent algorithm, which is applied to the detection method of S1 to S9 above, and includes a belt conveyor 1, a feeding hopper 2, a depth camera 3, and a weighing hopper 4.

[0101] The belt conveyor 1 includes a conveyor belt 1.1. The bracket of the belt conveyor 1 serves as the bracket of the entire device to install relevant components. The part of the conveyor belt 1.1 that bears the seeds to be measured is formed with a plurality of gratings 1.2 for accommodating the seeds to be measured. As Figure 4 and5 As shown, the purpose of setting the grid 1.2 is to facilitate the depth camera 3 to obtain a complete image of the seeds, so as to form a sequential detection image. That is to say, the seeds to be tested are contained in each grid 1.2, and the grid 1.2 is used as a boundary, so that the depth camera 3 can obtain a complete image including each grid 1.2 and the seeds to be tested contained therein. The grid 1.2 of the conveyor belt 1.1 in the prior art is composed of two side skirts and two front and rear baffles 1.3, and there is a gap between the skirt and the baffle 1.3. Therefore, this grid 1.2 belongs to an open grid 1.2. When using this grid 1.2 structure to contain the seeds to be tested, the seeds to be tested will fall into the gap, so that a complete image of the seeds cannot be obtained. In order to overcome the above defects, the present invention has improved the structure of the grid 1.2: a number of baffles 1.3 are equidistantly distributed on the conveyor belt 1.1. Both the baffle 1.3 and the conveyor belt 1.1 are made of rubber material, and the two are fixed by cementing. Elastic side plates 1.4 are arranged on both sides of the baffle 1.3. The elastic side plate 1.4 is a plastic thin plate or a metal thin plate, so as to have the ability of deformation and rebound. The elastic side plate 1.4 is an "L" - shaped structure body. One side structure body is fixed on both sides of the baffle 1.3 by cementing, and the other side structure body extends backward. The included angle between the two side structure bodies of the elastic side plate 1.4 is 92 - 95°. Horizontal guide rods 1.5 are arranged on the brackets on both sides of the conveyor belt 1.1. When the elastic side plate 1.4 is located below or at the front and rear ends, it remains in an outward - open state without being affected by the horizontal guide rod 1.5. When the elastic side plate 1.4 is located above and moves to the position of the horizontal guide rod 1.5, due to the extrusion of the horizontal guide rod 1.5, one side structure body of the elastic side plate 1.4 will be extruded to deform inward, and overlap on the outside of one side structure body of the elastic side plate 1.4 behind, so as to cooperate with the two front and rear baffles 1.3 to form a closed grid 1.2 to accommodate the seeds to be tested. In this way, after the grid 1.2 moves to the blanking position along with the conveyor belt 1.1, the seeds to be tested will fall into the grid 1.2, enabling the depth camera 3 in the subsequent link to obtain a complete image. When the grid 1.2 moves to the end, the seeds fall into the weighing hopper 4, and one side structure body of the elastic side plate 1.4 returns to its original state due to the loss of the extrusion of the horizontal guide rod 1.5.

[0102] Obviously, the grid 1.2 should be formed before the conveyor belt 1.1 reaches the landing point of the seeds to be tested, so as to prevent the seeds from appearing in other positions and affecting the detection. For the convenience of subsequent identification, the surface colors of the conveyor belt 1.1, the baffle 1.3 and the elastic side plate 1.4 should be the same or similar, and have a large color contrast with the seeds.

[0103] The blanking hopper 2 is installed on the bracket of the belt conveyor device 1 through a bracket and is located above the conveyor belt 1.1. A feed door 2.1 for periodically opening and closing the feed port is arranged at the bottom of the blanking hopper 2 through an opening and closing device 5.

[0104] In order to reduce the situation of overlapping after the seeds to be tested fall into the grille 1.2, the feeding hopper 2 uses vibratory feeding to make the seeds to be tested more dispersed. As Figure 4 、 Figure 11 and Figure 12 shown, the opening and closing device 5 is installed at the bottom of the feeding hopper 2, which includes an opening and closing motor 5.2 and an opening and closing cam 5.1. The opening and closing cam 5.1 is installed below the feed door 2.1 through the opening and closing cam 5.1, and the motor shaft of the opening and closing motor 5.2 is connected to the shaft of the opening and closing cam 5.1. The rim of the opening and closing cam 5.1 forms a sliding contact with the bottom surface of the feed door 2.1. As Figure 9 shown, when the opening and closing motor 5.2 drives the opening and closing cam 5.1 to rotate until its small-diameter surface contacts the bottom surface of the feed door 2.1, the feed door 2.1 is in the open state, and at this time the seeds to be tested will fall into the grille 1.2 through the gap. When the opening and closing cam 5.1 rotates until its large-diameter surface contacts the bottom surface of the feed door 2.1, the feed door 2.1 is in the closed state, and at this time the seeds to be tested cannot fall. During a feeding cycle, the feed door 2.1 opens and closes multiple times to produce the effect of vibratory feeding, so that the seeds to be tested are dispersed in the grille 1.2.

[0105] In order to further eliminate the situation of seed overlapping, as Figure 9 and Figure 10 shown, the device also includes a flattening and vibrating device 8, which is located below the area between the feeding area and the detection area of the conveyor belt 1.1, so as to provide a vibration force to this area. The flattening and vibrating device 8 includes a vibration motor 8.1, a vibration cam 8.2 and a vibration rod 8.3. The vibration cam 8.2 is directly driven by the vibration motor 8.1 to rotate, and the vibration rod 8.3 is vertically slidably installed, and its lower end is in sliding contact with the outer edge surface of the vibration cam 8.2. The high-speed rotation of the vibration cam 8.2 drives the vibration rod 8.3 to quickly strike the conveyor belt 1.1, so as to make it generate high-frequency and low-amplitude vibrations. In this way, after the conveyor belt 1.1 is vibrated, the seeds to be tested tend to be flattened, avoiding aggregation and overlapping. And the low amplitude avoids large gaps between the elastic side plate 1.4 and the surface of the conveyor belt 1.1, resulting in seed leakage. The magnitude of the vibration force can be adaptively adjusted according to the seed quality and distribution, which belongs to conventional technology and will not be elaborated here.

[0106] As Figure 11 and Figure 12 shown, since the feed door 2.1 may get stuck with seeds and cannot be normally closed when it is closed, for this reason, an elastic guide plate 2.2 is provided on the front wall of the feed port of the feeding hopper 2. After the feed door 2.1 is closed, it can squeeze the lower edge of the elastic guide plate 2.2 to deform it. After the feed door 2.1 is opened, the lower edge of the elastic guide plate 2.2 returns to its original state and forms a discharge channel with the feed door 2.1. By using the deformation ability of the elastic guide plate 2.2, when the feed door 2.1 is closed, it can squeeze the elastic guide plate 2.2 to deform inward, thus avoiding seed jamming.

[0107] To facilitate the discharge of all seeds, the feeding hopper 2 is installed obliquely and its adjustable installation is realized through the angle adjustment mechanism 7. When the inclination angle changes, the number of seeds discharged each time will also change accordingly. The angle adjustment mechanism 7 uses an electric push rod.

[0108] The depth camera 3 is installed on the bracket of the belt conveyor 1 through a bracket and is located above the conveying path. After the grille 1.2 moves to directly below the depth camera 3, the depth camera 3 captures images of the grille 1.2 and the seeds inside and transmits them to the control device. Since a plurality of grilles 1.2 are formed on the conveyor belt 1.1, the depth camera 3 sequentially obtains images of each grille 1.2. When the system detects each image, if no seeds are detected in a continuous plurality of images, a fault warning is issued. If the warning is ignored, it is considered that all the seeds in this batch have been detected.

[0109] The weighing hopper 4 is arranged below the end of the belt conveyor 1 to collect and detect the total weight of the seeds in this batch. The bottom of the weighing hopper 4 can be automatically opened after all the seeds in this batch have been detected to discharge the seeds. Since its structural part has not been improved, any weighing hopper 4 in the prior art can be used, so it will not be elaborated here. After all the seeds in this batch have been detected, the weighing hopper 4 will measure the total weight of the seeds in this batch, and the system will then calculate the thousand-grain weight of the seeds based on the number of seeds. A material guiding port 6 is also arranged between the weighing hopper 4 and the end of the conveyor belt 1.1 to facilitate the seeds to slide into the weighing hopper 4.

[0110] Since the system detects plump seeds and shriveled seeds, in subsequent processes, the plump seeds can also be screened out by a winnowing device for later use.

[0111] It should be noted that the electrical control method of this device is completed by the control device. The above intelligent seed testing system is deployed in the control device. After the system runs, it executes steps S1 to S9. The rest of the electrical control principles and processes belong to the prior art, and the present invention does not involve improvements in this part. When this device is running, it executes steps S1 to S9 to complete the seed detection process of one batch. Specifically as follows:

[0112] The seeds to be tested are introduced into the feeding hopper 2, and after selecting the seed type, the device is started. As the conveyor belt 1.1 moves, the grilles 1.2 are sequentially formed on the upper surface of the conveyor belt 1.1. At the same time, as the material gate 2.1 is repeatedly opened, the seeds successively fall into each grille 1.2. When the seeds in the grille 1.2 come under the depth camera 3 along with the conveyor belt 1.1, the depth camera 3 captures images of the seeds and sends them to the control device. The control device executes the detection process to complete the detection of the volume of the seeds in each grille 1.2, and finally identifies plump seeds, shriveled seeds and the quantities of both.

[0113] When the system continuously detects that multiple gratings 1.2 have no seeds, the system issues a fault warning. The staff determines whether there is a fault at this time based on the warning. If not, it is considered that all seed detections are completed. At this time, the system calculates the thousand-grain weight based on the weight and the number of seeds detected by the weighing hopper 4. Thus, the entire detection process ends.

[0114] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A seed detection method based on a multimodal intelligent algorithm, characterized in that: The steps include: S1, obtaining a grid image of the seeds to be tested; S2, collecting RGB-D modality information of the raster image; S3, cropping the edge portion of the raster image, and matching the cropped image with the RGB-D modality information; S4, using a detection algorithm to perform target detection on the image to determine a detection area of ​​the seed in the image; S5, establishing three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area; S6, using an adaptive density clustering algorithm guided by machine learning to segment the three-dimensional point cloud data into point cloud data corresponding to each seed; S6 includes the following sub-steps: S601, preprocessing and feature extraction processing are performed on the three-dimensional point cloud data to obtain the features of each point; S602: predicting the local density score of each point using a density prediction model according to the features; S603, adjusting key parameters of the DBSCAN clustering model according to the local density score; S604, substituting the key parameters and the three-dimensional point cloud data into the DBSCAN clustering model to obtain the point cloud data corresponding to each seed; S7. Calculate the volume data of each seed using a point cloud convex hull algorithm according to the point cloud data; S8. Use K-means clustering algorithm to cluster all volume data into solid particle data and deflated particle data.

2. The seed detection method based on a multimodal intelligent algorithm according to claim 1, characterized in that: The detection algorithm is the YOLOv8 algorithm or the YOLOv8-seg algorithm.

3. The seed detection method based on multimodal intelligent algorithm according to claim 2 is characterized in that: The detection algorithm is trained by the following steps: S401, marking the image to determine a detection area; S402, dividing the labeled images into a training set and a test set; S403, training the detection algorithm with the training set to obtain a training result; S404: Verify the training result using the test set, and obtain the optimal weights of seeds of different varieties after the training is completed.

4. The seed detection method based on a multimodal intelligent algorithm according to claim 1, characterized in that: It also includes S9, calculating the seed setting rate based on the total number of seeds, the data of filled seeds and the data of shrunken seeds; The step S4 further includes determining the total number of seeds according to the number of the detection areas.

5. A seed detection system based on a multimodal intelligent algorithm, applied to a seed detection method based on a multimodal intelligent algorithm as claimed in claim 1, characterized in that: include: An acquisition module is used to acquire a raster image of the seed to be tested; An acquisition module, used for acquiring RGB-D modality information of the raster image; A cropping module, used for cropping edge portions of the raster image and matching the cropped image with the RGB-D modality information; A detection module, used to perform target detection on the image using a detection algorithm to determine a detection area of ​​the seed in the image; A three-dimensional point cloud data acquisition module, used to establish three-dimensional point cloud data according to the RGB-D modality information of the pixel points in the detection area; A clustering module, used for segmenting the three-dimensional point cloud data into point cloud data corresponding to each seed by using an adaptive density clustering algorithm guided by machine learning; A volume calculation module, used to calculate the volume data of each seed using a point cloud convex hull algorithm according to the point cloud data; The K-means clustering module is used to cluster all volume data into solid particle data and deflated particle data using the K-means clustering algorithm.

6. A seed detection system based on a multimodal intelligent algorithm according to claim 5, characterized in that: It also includes a fruiting rate calculation module, which is used to calculate the fruiting rate based on the total number of seeds, the data of filled grains and the data of shrunken grains; The detection module further performs a step of determining a total number of seeds according to the number of the detection areas.

7. A seed detection device based on a multimodal intelligent algorithm, characterized in that: include: A belt conveyor device (1), the belt conveyor device (1) comprising a conveyor belt (1.1), a portion of the conveyor belt (1.1) carrying the seeds to be tested being formed with a plurality of grids (1.2) for accommodating the seeds to be tested; A material discharge hopper (2), the material discharge hopper (2) being arranged above the material discharge position of the belt conveyor device (1), and having a material gate (2.1) at its bottom for realizing periodic opening and closing of a material opening through an opening and closing device (5); A depth camera (3), the depth camera (3) being arranged above the conveying path of the belt conveyor device (1); A weighing hopper (4), the weighing hopper (4) being arranged below the end of the belt conveyor (1) to collect and detect the total weight of the batch of seeds; A control device, wherein the control device executes the steps of a seed detection method based on a multimodal intelligent algorithm as described in claim 1.

8. A seed detection device based on a multimodal intelligent algorithm according to claim 7, characterized in that: The lower hopper (2) is adjustable via an angle adjustment mechanism (7); The opening and closing device (5) is installed at the bottom of the lower hopper (2), and comprises an opening and closing motor (5.2) and an opening and closing cam (5.1); the opening and closing cam (5.1) is installed below the material door (2.1) through the opening and closing cam (5.1), and the motor shaft of the opening and closing motor (5.2) is connected to the shaft of the opening and closing cam (5.1); the wheel rim of the opening and closing cam (5.1) forms a sliding contact with the bottom surface of the material door (2.1); An elastic material guide plate (2.2) is provided on the front wall of the material opening of the lower hopper (2); when the material door (2.1) is closed, the lower edge of the elastic material guide plate (2.2) can be pressed inwards to deform the lower edge; when the material door (2.1) is opened, the lower edge of the elastic material guide plate (2.2) returns to its original shape and forms a material discharge channel between the material door (2.1).

9. The seed detection device based on multimodal intelligent algorithm according to claim 7, characterized in that: It also comprises a flattening vibration device (8), wherein the flattening vibration device (8) is located below the area between the unloading area and the detection area of ​​the conveyor belt (1.1), thereby providing a vibration force to the area.

10. The seed detection device based on multimodal intelligent algorithm according to claim 7, characterized in that: A plurality of baffles (1.3) are evenly spaced on the conveyor belt (1.1), and elastic side plates (1.4) are arranged on both sides of the baffles (1.3); horizontal guide rods (1.5) are arranged on both sides of the conveyor belt (1.1); and the elastic side plates (1.4) overlap the outer sides of the elastic side plates (1.4) at the rear under the action of the horizontal guide rods (1.5) as the conveyor belt (1.1) moves forward, thereby forming a grid (1.2) for accommodating seeds to be tested between the front and rear baffles (1.3) and the left and right elastic side plates (1.4).

Images