Seed multi-parameter phenotype detection device and detection method thereof

Abstract

The invention provides a seed multi-parameter phenotype detection device and method, and the device comprises a mechanical arm which is used for sucking seeds; the camera I is used for acquiring pictures of the seeds at all angles during rotation; the culture dish is used for packaging and culturing seeds; the chain conveyor is used for conveying the culture dishes containing the seeds; the water supplementing assembly is used for supplementing water to the seeds in the culture dish; the image acquisition module comprises a camera II, a laser ranging module and a photoelectric sensor II; and the computer is connected with the camera I and the image acquisition module and is used for completing three-dimensional reconstruction of the seeds, realizing extraction of phenotypic parameters of seed cotyledons, extraction of seed root systems, restoration of broken roots and establishment of a seed activity prediction model. Therefore, real-time dynamic nondestructive testing of morphological characteristics (structure, color, size and the like) and biochemical characteristics (vitality index and the like) of the seeds is realized, so that a basic information database and a vitality prediction model of the soybean seeds are established.

Description

Technical Field

[0001] The present invention relates to the technical field of seed quality detection, and in particular to a seed multi-parameter phenotype detection device and a detection method thereof. Background Art

[0002] Seed testing is an important agricultural process that aims to evaluate the quality, purity, vigor and health of seeds. Seed testing can ensure that the seeds planted have high germination rates, good growth performance and disease resistance.

[0003] The traditional identification methods of soybean seed detection technology currently available in China mainly include manual means, computer vision technology and chemical methods. Manual means mainly detect the external quality of soybeans based on expert experience. Not only is it time-consuming, but it also cannot guarantee the objectivity and accuracy of identification. Computer vision method is an effective method for rapid and non-destructive identification of seed varieties based on external feature differences. However, since the appearance of different varieties of soybeans is very similar, computer vision technology is not suitable for identifying soybean varieties, and this method cannot detect the intrinsic quality of soybean seeds. Chemical methods are also a powerful means of identifying seeds. The method of using chemical means to detect the intrinsic quality of soybeans requires grinding soybean seeds and obtaining test results through chemical analysis. Using chemical methods for detection not only has destructive and contaminating problems, but also has time-consuming problems. The inspection of seed variety identification and vitality grade detection mainly relies on manual operation by seed inspectors, with a long work cycle and low accuracy, which cannot meet the needs of the rapid development of the current seed market.

[0004] With the continuous development and technological progress of the domestic seed industry, the application of seed phenotypic trait assembly lines is also gradually increasing. For example, the "intelligent seed phenotypic detection and seedling growth monitoring system" developed by the Beijing Agricultural Seed Science Research Institute and the fully automatic seed density measurement equipment developed by the Jiangsu Seed Quality Inspection Center. But in general, there are still certain gaps and shortcomings in the research and application of seed phenotypic trait assembly lines in China. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention aims to provide a seed multi-parameter phenotype detection device and a detection method thereof, which can solve the existing traditional problems.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] The present invention is achieved through the following technical solution: The present invention provides a seed multi-parameter phenotype detection device, comprising:

[0008] A mechanical arm, wherein the mechanical arm is used to suck the seeds and rotate the seeds;

[0009] Camera 1, the camera 1 is arranged on the side of the mechanical arm, and the camera 1 is used to obtain pictures of seeds at various angles during rotation;

[0010] A culture dish, which is used to collect and culture seeds;

[0011] A chain conveyor, wherein the chain conveyor is used to convey the culture dish containing the seeds;

[0012] A water replenishing component, which is used to replenish water for the seeds in the culture dish;

[0013] An image acquisition module, the image acquisition module includes a second camera, a laser ranging module and a second photoelectric sensor; the second photoelectric sensor is used to sense the culture dish, and the second camera collects seed images of different time periods according to changes in the laser ranging module;

[0014] A computer is connected to camera 1 and an image acquisition module; the computer completes three-dimensional reconstruction of seeds based on the pictures acquired by camera 1, obtains its morphological characteristic parameters and establishes a relevant database; the computer uses a deep learning algorithm based on the image acquisition module to extract seed cotyledon phenotypic parameters, extract and repair seed roots, and establish a seed vitality prediction model.

[0015] Furthermore, the robotic arm includes a support plate, a pneumatic guide rail, a pneumatic slider, an axis arm one, an axis arm two, an axis arm three, a robotic arm gripper, a suction nozzle and a motor; a pneumatic guide rail is installed on the top of the support plate, and a pneumatic slider is slidably connected to the side of the pneumatic guide rail; an axis arm one is fixed to the side of the pneumatic slider, and a motor is provided at the end of the axis arm one; the output end of the motor is connected to the axis arm two, and a motor is provided at the end of the axis arm two; the output end of the motor is connected to the axis arm three, and a motor is provided at the end of the axis arm three; the output end of the motor is connected to the robotic arm gripper, and the robotic arm gripper clamps the suction nozzle; the other end of the suction nozzle is connected to a negative pressure vacuum pump.

[0016] Furthermore, the water replenishment component includes a water tank, a nozzle, a photoelectric sensor 1, a water pump, a mounting seat, a single-chip microcomputer, a support plate and a relay; a single-chip microcomputer and a relay are provided on the side of the mounting seat, and a support plate is provided at the top of the mounting seat; a water tank is provided on the top of the support plate, and a water pump is provided inside the water tank; the output end of the water pump is connected to the nozzle through a water pipe, and a photoelectric sensor 1 is provided on the side of the support plate; the photoelectric sensor 1, the water pump and the relay are connected to the single-chip microcomputer; the photoelectric sensor 1 and the relay are used to control the switch of the water pump; when the culture dish passes through the photoelectric sensor 1, the photoelectric sensor 1 sends a signal to the single-chip microcomputer for processing, and after receiving the signal, the single-chip microcomputer sets the IN terminal to a high level, controls the relay to be attracted, and then controls the water pump switch.

[0017] Furthermore, the camera is used to obtain pictures of the seeds at various angles during rotation, including the robotic arm sucking the seeds and rotating them. The camera takes pictures of the seeds after one rotation. Between each picture, the seeds rotate 15°, for a total rotation of 360°. A single seed requires 24 pictures.

[0018] Furthermore, the three-dimensional reconstruction of the seeds includes:

[0019] Get multi-angle images of the seeds and ensure that the image sizes are consistent;

[0020] Preprocessing the acquired multi-angle images;

[0021] Calibrate camera one and perform contour extraction;

[0022] Perform fixed-point connection and texture mapping for the surface of the 3D model;

[0023] Perform three-dimensional reconstruction of seeds.

[0024] Furthermore, the extraction of seed cotyledon phenotypic parameters includes: using the YOLOv7-OBB recognition algorithm; specifically:

[0025] YOLOv7-OBB uses a rotated rectangular box as an anchor box; each label contains x, y (horizontal and vertical coordinates of the center of the ellipse), rx, ry (the major and minor axes of the ellipse), and θ (the rotation angle of the ellipse); the ellipse label is converted into a minimum circumscribed rectangular label with a rotation angle; the YOLOv7-OBB network BackBone model is CSPDarknet, which is used to extract features from the input image; the PANet structure is used as an enhanced feature extraction network of the model to fuse features from different levels of CSPDarknet; the feature points are judged by the YOLO Head in the prediction network to generate a detection box and the corresponding target category; the rotation angle is added to the output of the Yolo-Head of YOLOv7-OBB; KLD_Loss is used as the loss function;

[0026] The images of seeds in different time series are input into the YOLOv7-OBB model in real time. The model recognizes the seeds and outputs the anchor frame parameters, which are sorted and numbered according to the center point coordinates. The rectangular anchor frame is converted into a minimum inscribed ellipse, and the seed elliptical anchor frame is mapped to a blank image of the same size as the original image, where the pixel value of the elliptical anchor frame area is set to 255 and the background pixel value is 0, that is, a single seed binary image. The images of different time series are preprocessed by image enhancement and contrast enhancement, and then the images are converted from RGB to HSV color space, and the color features are used to implement threshold segmentation to obtain a binary image. Combined with the center point coordinates of the anchor frame, the single seed binary image is extracted according to the region growing algorithm. The length, width, aspect ratio, area, perimeter and circularity shape characteristics corresponding to the single seed are detected by contour extraction, which are the phenotypic parameters of the seed in the absorption and expansion state.

[0027] Furthermore, the seed root extraction and root cutting repair comprises:

[0028] The number of connected regions N is used to determine whether a single seed image has broken roots, and N-2 is used as the number of times to repair broken roots. Under the eight-connected domain, the center point of one adjacent skeleton point is defined as an endpoint, and this point is defined as a common point.

[0029] Traverse the skeleton graph of the connected area in sequence, extract the endpoint coordinates respectively and sort them by Y-axis coordinates, and extract the minimum and maximum points on the Y-axis; due to the geotropism of the root, the minimum and maximum points on the Y-axis are taken as the root tip and root tail points according to the position characteristics, and the remaining points are the breakpoints;

[0030] According to the order of the breakpoints, two breakpoints are taken in turn for second-order polynomial fitting until the number of root repairs is reached;

[0031] The single seed binary image is repaired with the average width of two connected areas; in order to calculate the length and width of the main root, the lateral roots in the binary image are pruned by the lateral root separation algorithm of the contour; unlike the thinned skeleton image, there is no determination of endpoints and intersections in the binary image; the Douglas-Peuker algorithm is used to detect the feature points in the contour and number the feature points counterclockwise; by observing the root growth rule, the lateral root tail is a convex point, and the intersection with the main root is a concave point. The separated lateral roots can be connected according to the two concave points near the convex point of the lateral root; therefore, since the feature point at the root tip must be a convex point, this point is used as the reference point; the convexity of the feature point is judged by the cross product result; if the coordinates of the feature point at the root tip are (x0, y0), and the coordinates of the current feature point are (x1, y1), then the cross product formula is:

[0032] cross=x0×y1-x1×y0

[0033] The sign of the cross product is used to determine whether it is a convex point or a concave point; after determining the convex point, the angle between the convex point and the previous feature point and the next feature point is calculated, and the convex points whose angles are close to the straight line are removed; thus, the number of lateral roots is determined based on the number of convex points; thus, the lateral roots are separated by connecting the previous concave point and the next concave point of the convex point.

[0034] Furthermore, the establishment of a seed vitality prediction model comprises:

[0035] The vitality prediction model adopts a long short-term memory (LSTM) model; Adam is used to train the LSTM model, and it is trained in small batches of size 16, with a learning rate of 0.001 and 100 epochs. The LSTM model consists of a 64-neuron LSTM layer and a 32-neuron LSTM layer, and a dropout regularization of 0.2 is applied to prevent the model from overfitting; the LSTM model uses mean square error (MSE), root mean square error (RMSE) and R-squared indicators to evaluate the performance and prediction accuracy of the model.

[0036] A seed multi-parameter phenotypic detection method, comprising:

[0037] A mechanical arm is used to suck the seeds and rotate them;

[0038] Using camera 1 to obtain pictures of seeds at various angles during rotation, and completing three-dimensional reconstruction of the seeds based on the pictures obtained by camera 1, obtaining its morphological characteristic parameters and establishing a relevant database;

[0039] A chain conveyor is used to transport the culture dish containing the seeds, and a water supply component is used to supply water to the seeds in the culture dish;

[0040] Camera 2 is used to collect seed images in different time periods according to the changes in the laser ranging module. The deep learning algorithm is used according to the image acquisition module to extract the phenotypic parameters of seed cotyledons, extract and repair the root system of seeds, and establish a seed vitality prediction model.

[0041] Compared with the prior art, the beneficial effects of the present invention include:

[0042] The present invention discloses a seed multi-parameter phenotype detection device and a detection method thereof. First, the SCARA robot arm is controlled to suck soybean seeds, and a camera is used to capture an image of the seeds rotating for one week. At the same time, a silhouette contour extraction method is used to perform three-dimensional reconstruction of the seed phenotype, and its morphological properties are obtained. Then, the seed is placed in a culture dish on an assembly line, and a nutrient solution is dripped into the culture dish at regular intervals to perform seed germination culture. Finally, a deep learning technique is used to separate cotyledons from seedlings, separate taproots from lateral roots, and repair broken roots, calculate root growth parameters, perform vitality assessment according to dynamic time series data, and establish a vitality prediction model through LSTM. Therefore, the present invention realizes real-time dynamic non-destructive detection of morphological characteristics (structure, color, size, etc.) and biochemical characteristics (vitality index, etc.) of seeds, thereby establishing a basic information database of soybean seeds and a vitality prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. Among them:

[0044] Figure 1 It is an overall schematic diagram of a seed multi-parameter phenotype detection device in an embodiment of the present invention;

[0045] Figure 2 It is a schematic diagram of the structure of the mechanical arm in the implementation case of the present invention;

[0046] Figure 3 It is a bottom logic diagram of the robot arm control program in the implementation case of the present invention;

[0047] Figure 4 It is a structural schematic diagram of the water replenishment component in the implementation case of the present invention;

[0048] Figure 5 It is a schematic diagram of the seed three-dimensional reconstruction process in the implementation case of the present invention;

[0049] Figure 6 A schematic diagram of calibration of a calibration pad in three-dimensional reconstruction in an implementation case of the present invention;

[0050] Figure 7 A diagram showing the three-dimensional reconstruction process of soybean seeds in an implementation example of the present invention;

[0051] Figure 8 It is a schematic diagram of numbering characteristic points in the root system repair algorithm in the implementation case of the present invention;

[0052] Fig. 9 A schematic diagram of extracting root system information using a root system repair algorithm in an implementation case of the present invention;

[0053] Fig.10 The figure is a schematic diagram of the process of the multi-parameter seed phenotyping method in the implementation case of the present invention.

[0054] As shown in the figure:

[0055] 1. Chain conveyor; 2. Culture dish; 3. Water replenishment component; 31. Water tank; 32. Nozzle; 33. Photoelectric sensor 1; 34. Water pump; 35. Mounting base; 36. Single chip microcomputer; 37. Pallet; 38. Relay; 4. Storage box; 5. Robotic arm; 51. Support plate; 52. Pneumatic guide rail; 53. Pneumatic slider; 54. Axis arm 1; 55. Axis arm 2; 56. Axis arm 3; 57. Robotic arm gripper; 58. Suction nozzle; 59. Motor; 6. Camera 1; 7. Image acquisition module. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of this document and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this document described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0058] This specification provides method operation steps as described in the embodiments or flow charts, but more or fewer operation steps may be included based on routine or non-creative work. The order of steps listed in the embodiments is only one way of executing the order of many steps and does not represent the only execution order. When the system or device product is executed in practice, it can be executed in the order of the method shown in the embodiments or the drawings or in parallel.

[0059] The present invention provides a seed multi-parameter phenotype detection device, illustratively, taking soybean seed detection as an example; Figure 1 Shown, including

[0060] A storage box 4, wherein the storage box 4 is used to store manually selected seeds;

[0061] The mechanical arm 5 is used to suck the seeds in the storage box 4 and rotate the seeds; the mechanical arm 5 includes a support plate 51, a pneumatic guide rail 52, a pneumatic slider 53, an axis arm 1 54, an axis arm 2 55, an axis arm 3 56, a mechanical arm gripper 57, a suction nozzle 58 and a motor 59; Figure 2 As shown, a pneumatic guide rail 52 is installed at the top of the support plate 51, and a pneumatic slider 53 is slidably connected to the side of the pneumatic guide rail 52; a shaft arm 1 54 is fixed to the side of the pneumatic slider 53, and a motor 59 is provided at the end of the shaft arm 1 54; the output end of the motor 59 is connected to a shaft arm 2 55, and a motor 59 is provided at the end of the shaft arm 2 55; the output end of the motor 59 is connected to a shaft arm 3 56, and a motor 59 is provided at the end of the shaft arm 3 56; the output end of the motor 59 is connected to a robot arm gripper 57, and the robot arm gripper 57 clamps a suction nozzle 58; the other end of the suction nozzle 58 is connected to a negative pressure vacuum pump.

[0062] Exemplarily, the robot arm 5 used in the present invention is preferably a SCARA robot arm, which is a four-axis collaborative robot arm with built-in drive control, and has the characteristics of high precision, large arm span, light weight, and high safety. The product weight is about 11kg, the maximum load can reach 111kg, the rotation angle of the 1-axis is +90°, the rotation angle of the 2-axis is +143°, the rotation range of the R-axis can achieve +1080°, there are four degrees of freedom, and the linear speed is 1017mm / s. It is suitable for deployment in multiple applications, and can quickly switch between operating procedures and complete small batch production, etc.;

[0063] This SCARA robot arm has four degrees of freedom. It can collaborate with humans without safety protection devices to reduce repetitive strain and accidental injuries, and it can also be directly controlled by a host computer to write a program to perform corresponding actions.

[0064] The underlying logic of its control program is shown in Figure 3. By setting the target's spatial x, y, and z coordinates, it can accurately reach the target seed position and achieve seed absorption, rotation (controllable angle) and placement. By controlling the rotation speed of its four axes, it can achieve precise coordination with the assembly line, reduce unnecessary detection time, and improve detection efficiency.

[0065] A camera 6, which is disposed on the side of the mechanical arm 5 and is used to obtain pictures of the seeds at various angles during rotation;

[0066] A culture dish 2, wherein the culture dish 2 is used to collect and culture seeds;

[0067] Chain conveyor 1, the chain conveyor 1 is used to convey the culture dish 2 of the containerized seeds; the chain conveyor 1 is a production line system; it includes power equipment (motor), transmission shaft, roller, tensioning equipment, sprocket, chain, bearing, lubricant, chain plate, etc., wherein the two main parts driving the conveying of materials are chain and metal plate, the reciprocating motion of the chain provides traction power, and the metal plate serves as a carrier in the conveying process; its principle is to use a series of chains fixed on the traction chain to provide traction force, and use the metal plate as a carrier to convey the material in the horizontal direction. Its basic parameters are shown in Table 1.

[0068] Table 1 Basic parameters of the pipeline

[0069]

[0070] The RV reduction motor used in this assembly line system is composed of gears and turbines. It has a compact structure, a large transmission ratio, and has the basic function of locking under certain circumstances. RV reduction motors are generally used for low-speed and high-torque motor equipment. The output power of a gas turbine or other high-speed operation is achieved by the transmission gear on the input shaft of the reduction motor meshing with the large gear on the output shaft to achieve the deceleration effect. Generally, the reduction motor will have several pairs of transmission gears with the same mechanism to achieve the most ideal actual deceleration effect. The reduction motor is a power transmission mechanism that uses the speed ratio converter of the gear shaft to reduce the speed of the motor to the required speed and obtain greater torque. It is controlled by the HF020 controller and has built-in multiple fixed operating modes. Users can quickly select the appropriate motion trajectory, and can save and set the distance / speed / delay / number of cycles when the power is off. It can be operated independently as a module, or it can be used in conjunction with other systems as a third-party module to make up for the lack of motor output shafts.

[0071] The SCARA robot arm 5 is responsible for sucking the soybean seeds from the storage box 4 and transferring them to the culture dish 32 of the assembly line. During the transfer process, it is also necessary to take photos of the soybean seeds for a week. Between each photo, the soybean seeds rotate 15°, a total of 360°, and a single seed requires 24 photos.

[0072] The water replenishment component 3 is used to replenish water for the seeds in the culture dish; the water replenishment component 3 includes a water tank 31, a nozzle 32, a photoelectric sensor 33, a water pump 34, a mounting seat 35, a single chip microcomputer 36, a support plate 37 and a relay 38; Figure 4As shown, a single-chip microcomputer 36 and a relay 38 are provided on the side of the mounting seat 35, and a support plate 37 is provided at the top of the mounting seat 35; a water tank 31 is provided on the top of the support plate 37, and a water pump 34 is provided inside the water tank 31; the output end of the water pump 34 is connected to the nozzle 32 through a water pipe, and a photoelectric sensor 33 is provided on the side of the support plate 37; the photoelectric sensor 33, the water pump 34, and the relay 38 are connected to the single-chip microcomputer 36; the photoelectric sensor 33 and the relay 38 are used to control the switch of the water pump 34, and when a culture dish 32 passes through the photoelectric sensor 33, the water pump is turned on to provide a suitable environment for seed germination; when the culture dish 32 passes through the photoelectric sensor 33, the photoelectric sensor 33 sends a signal to the single-chip microcomputer 36 for processing, and after receiving the signal, the single-chip microcomputer 36 sets the IN terminal to a high level, controls the relay 38 to be attracted, so that the NO interface and the COM interface are short-circuited; and then the switch of the water pump 34 is controlled.

[0073] Image acquisition module 7, the image acquisition module 7 includes camera 2, laser ranging module and photoelectric sensor 2; the photoelectric sensor 2 is used to sense the culture dish, and the camera 2 collects seed images of different time periods according to the changes of the laser ranging module;

[0074] A computer is connected to camera 1 and an image acquisition module; the computer completes three-dimensional reconstruction of seeds based on the pictures acquired by camera 1, obtains its morphological characteristic parameters and establishes a relevant database; the computer uses a deep learning algorithm based on the image acquisition module to extract seed cotyledon phenotypic parameters, extract and repair seed roots, and establish a seed vitality prediction model.

[0075] Specifically, the three-dimensional reconstruction of the seed includes: obtaining multi-angle images of the seed and ensuring that the image sizes are consistent; preprocessing the obtained multi-angle images; calibrating the camera and extracting the contour; performing fixed-point connection and texture mapping for the three-dimensional model surface; and performing three-dimensional reconstruction of the seed. The process is as follows: Figure 5 shown.

[0076] The three-dimensional reconstruction technology adopted by the present invention is formed by combining the silhouette method with 3DSOM. The silhouette is the outline of the image of the object projected on the imaging plane, and is an important clue to understanding the geometric shape of the object. When a spatial target object is observed from multiple perspectives in a perspective projection manner, a silhouette outline of the object can be obtained in the corresponding picture of each perspective; here, the silhouette outline and the corresponding perspective projection center jointly determine a cone of a general shape in the three-dimensional space, and the observed object is located inside this three-dimensional cone. By analogy, by continuously increasing the number of observation perspectives of the target object from different directions, the shape of each corresponding cone can be continuously approached to the surface of the target object, thereby visualizing the shape characteristics of the target object in three dimensions. The above-mentioned spatial envelope formed by the intersection of the cones is called the visible shell of the object.

[0077] In order to obtain a visible shell that is closer to the actual surface of the object and make the three-dimensional modeling effect more accurate, it is often necessary to start from multiple directions and obtain a large number of images corresponding to viewing angles; and when shooting multi-angle images, it is necessary to rotate the object or rotate the camera to achieve the acquisition of pictures in different orientations, thereby changing the direction and pitch angle of the viewing angle corresponding to the target object; and in the later stage of three-dimensional modeling, it is necessary to establish a connection and combine the images of different directions and viewing angles. Therefore, the relationship between the spatial positions of each image is particularly important. In order to achieve this important step, it is necessary to perform "camera calibration" on the acquired multi-angle images to determine the camera shooting orientation and its internal related parameters corresponding to each image. The present invention adopts an auxiliary camera calibration method of a calibration device, and uses a calibration pad to determine the shooting orientation of each multi-angle image. The calibration pad is such as Figure 6 shown.

[0078] Finally, it is necessary to perform texture mapping on the surface of the three-dimensional model based on the corresponding orientation information characteristics of the three-dimensional surface contour model obtained above, combined with the orientation information of different multi-angle images, so that the model has more visual features and better describes the characteristics of the actual object. After executing automatic texture mapping, there may be some surface areas of the target object that still have no texture or the texture has obvious jumps. This is mostly caused by the loss of relevant information when some angles are not captured when multi-angle images are taken. To address this problem, with the help of corresponding images, manual texture matching can be performed in the direction where the relevant texture information is lost to supplement the texture information of the details. The three-dimensional reconstruction process is like this. Figure 7 shown.

[0079] The method for extracting the phenotypic parameters of seed cotyledons includes: adopting the YOLOv7-OBB recognition algorithm; specifically, YOLOv7-OBB uses a rotated rectangular frame as an anchor frame; each label includes x, y (horizontal and vertical coordinates of the center point of the ellipse), rx, ry (the major axis and the minor axis of the ellipse), and θ (the rotation angle of the ellipse); converting the ellipse label into a minimum circumscribed rectangular label with a rotation angle; the YOLOv7-OBB network BackBone model is CSPDarknet, which is used to extract features from an input image; using a PANet structure as an enhanced feature extraction network of the model, which is used to fuse features of different levels from CSPDarknet to further enhance the feature extraction capability of the model; judging feature points by a YOLO Head in a prediction network, generating a detection frame and a corresponding target category; adding a rotation angle to the output of the Yolo-Head of YOLOv7-OBB; and adopting KLD_Loss as a loss function;

[0080] The images of seeds in different time series are input into the YOLOv7-OBB model in real time. The model recognizes the seeds and outputs the anchor frame parameters, which are sorted and numbered according to the center point coordinates. The rectangular anchor frame is converted into a minimum inscribed ellipse, and the seed elliptical anchor frame is mapped to a blank image of the same size as the original image, where the pixel value of the elliptical anchor frame area is set to 255 and the background pixel value is 0, that is, a single seed binary image. The images of different time series are preprocessed by image enhancement and contrast enhancement, and then the images are converted from RGB to HSV color space, and the color features are used to implement threshold segmentation to obtain a binary image. Combined with the center point coordinates of the anchor frame, the single seed binary image is extracted according to the region growing algorithm. The shape characteristics of the single seed, such as length, width, aspect ratio, area, perimeter, and circularity, are detected by contour extraction, which are the phenotypic parameters of the seed in the absorption and expansion state.

[0081] The seed root system extraction and broken root repair include: in the soybean germination image, the cross-sectional width of the cotyledon is larger than that of the seedling, and the cotyledon is intercepted according to this feature. However, during the growth of soybeans, the cotyledon and the seedling may be adhered to each other. Therefore, the skeleton is extracted by a refinement algorithm, and then the image after hole filling is subtracted from the original skeleton image to determine whether the cotyledon and the seedling are adhered to each other. If adhesion occurs, a binary root system image is obtained by subtracting a single soybean binary image from a single seed binary image obtained by deep learning. If no adhesion occurs, the single soybean binary image is subjected to edge detection and skeleton extraction respectively, and the width is calculated by traversing the pixel points on the skeleton line using the orthogonal skeleton line method, and the pixel coordinates are used as keys and the width values ​​are put into a dictionary as values. Then the values ​​in the dictionary are sorted to take out the maximum and minimum values, and the threshold is set to:

[0082] threshold=(width max +width min ) / 2

[0083] The values ​​greater than the threshold are defined as the width of the cotyledon and the pixels are set to 0, and the values ​​less than the threshold are defined as the width of the seedling.

[0084] In order to extract seedling phenotypic parameters, the present invention proposes a root-breaking repair algorithm. First, the number of connected regions N is used to determine whether a single soybean image has root-breaking, and N-2 is used as the number of root-breaking repairs. Under the eight-connected domain, the center point of one adjacent skeleton point is defined as an endpoint, and this point is defined as a common point. Then, the skeleton graph of the connected region is traversed in sequence, such as Figure 8 As shown, the endpoint coordinates are extracted respectively and sorted by Y-axis coordinates, and the minimum and maximum points on the Y-axis are extracted. Due to the geotropism of the root, the minimum and maximum points on the Y-axis are used as the root tip and root tail points according to the position characteristics of the obtained coordinate points, and the remaining points are the breakpoints. Then, according to the order of the breakpoint arrangement, two breakpoints are taken in turn for second-order polynomial fitting until the number of broken roots is repaired. Finally, the single soybean binary image is repaired with the average width of the two connected areas. In order to calculate the length and width of the main root, the lateral roots in the binary image are pruned based on the contour-based lateral root separation algorithm of the present invention. Unlike the refined skeleton image, there is no determination of endpoints and intersections in the binary image. The Douglas-Peuker algorithm is used to detect the feature points in the contour and number the feature points counterclockwise, such as Fig. 9 As shown. By observing the root growth rules, the lateral root tail is a convex point, and the intersection with the main root is a concave point. The separated lateral roots can be connected by connecting the two concave points near the convex point of the lateral root. Therefore, since the feature point at the root tip must be a convex point, this point is used as the reference point. The convexity of the feature point is judged by the cross product result. Suppose the coordinates of the feature point at the root tip are (x0, y0), and the coordinates of the current feature point are (x1, y1), then the cross product formula is:

[0085] cross=x0×y1-x1×y0

[0086] The sign of the cross product is used to determine whether it is a convex point or a concave point. After determining the convex point, the angle between the convex point and the previous feature point and the next feature point is calculated, and the convex points whose angles are close to the straight line are removed. The number of lateral roots is thus determined based on the number of convex points. Thus, the lateral roots are separated by connecting the previous concave point of the convex point and the next concave point.

[0087] The seed vitality prediction model is established, and the prediction vitality prediction model adopts a long short-term memory (LSTM) model, which is a special type of recursive neural network that can predict according to each time step of the sequence as the network input. The present invention uses Adam to train the LSTM model, and trains it in a small batch of 16 sizes, with a learning rate of 0.001, 100 epochs, and the model consists of a LSTM layer of 64 neurons and a LSTM layer of 32 neurons, and a dropout regularization of 0.2 is applied to prevent the model from overfitting. The LSTM model uses indicators such as mean square error (MSE), root mean square error (RMSE) and R square value (R-squared) to evaluate the performance and prediction accuracy of the model. In the present invention, MSE is 0.00047, RMSE is 0.02167, and R-squared is 0.99. For similar research problems, the mapping of plant phenotype-genotype by LSTM model is carried out according to the dynamic behavior in the growth process of plants. Using the dynamic shape of plant growth as an important discriminant phenotype, the LSTM model uses the correlation between continuous image phenotypic parameters to model features compared to other models, which can achieve high-precision prediction or classification.

[0088] The present invention also provides a seed multi-parameter phenotype detection method, the process of which is as follows: Fig.10 As shown, including:

[0089] A mechanical arm is used to suck the seeds and rotate them;

[0090] Camera 1 is used to obtain pictures of seeds at various angles during rotation, and based on the pictures obtained by camera 1, three-dimensional reconstruction of the seeds is completed to obtain its morphological characteristic parameters and establish a relevant database; specifically, multi-angle images are obtained, and if the image sizes are consistent, image preprocessing, noise removal and distortion correction are performed; then contour extraction and texture mapping are performed, and then a three-dimensional model of soybean seeds is constructed to establish a seed morphological characteristic database.

[0091] A chain conveyor is used to transport the culture dish containing the seeds, and a water replenishing component is used to replenish water to the seeds in the culture dish; the water replenishment is carried out at a fixed time.

[0092] Camera 2 is used to collect seed images in different time periods according to the changes in the laser ranging module. The deep learning algorithm is used according to the image acquisition module to extract the phenotypic parameters of seed cotyledons, extract and repair the root system of seeds, and establish a seed vitality prediction model.

[0093] The present invention is based on three-dimensional reconstruction technology, spatial frequency domain imaging technology and deep learning algorithm, combined with single-chip microcomputer, assembly line and SCARA mechanical arm to build an automatic detection system for obtaining soybean seed phenotypic parameters, realize real-time dynamic non-destructive detection of morphological characteristics (structure, color, size, etc.) and biochemical characteristics (vitality index, etc.) of seeds, thereby establishing a basic information database of soybean seeds and a vitality prediction model. The basic detection process is to first control the SCARA mechanical arm to suck soybean seeds, and use a camera to take an image of the seeds rotating for one week, and use the silhouette contour extraction method to perform three-dimensional reconstruction of the seed phenotype, obtain its morphological traits, and then place it in a culture dish on the assembly line, and regularly drip nutrient solution into the culture dish for seed germination and cultivation, and finally use deep learning technology to achieve separation of cotyledons and seedlings, separation of taproots and lateral roots, and root repair, calculate root growth parameters, perform vitality evaluation according to dynamic time series data, and establish a vitality prediction model through LSTM.

[0094] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this article.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

[0098] In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0099] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0100] Specific embodiments are used in this article to illustrate the principles and implementation methods of this article. The description of the above embodiments is only used to help understand the methods and core ideas of this article. At the same time, for general technicians in this field, according to the ideas of this article, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on this article.

Claims

1. A seed multi-parameter phenotype detection device, characterized in that: include A mechanical arm, wherein the mechanical arm is used to suck the seeds and rotate the seeds; Camera 1, the camera 1 is arranged on the side of the mechanical arm, and the camera 1 is used to obtain pictures of seeds at various angles during rotation; A culture dish, which is used to collect and culture seeds; A chain conveyor, wherein the chain conveyor is used to convey the culture dish containing the seeds; A water replenishing component, which is used to replenish water for the seeds in the culture dish; An image acquisition module, the image acquisition module includes a second camera, a laser ranging module and a second photoelectric sensor; the second photoelectric sensor is used to sense the culture dish, and the second camera collects seed images of different time periods according to changes in the laser ranging module; A computer is connected to camera 1 and an image acquisition module; the computer completes three-dimensional reconstruction of seeds based on the pictures acquired by camera 1, obtains its morphological characteristic parameters and establishes a relevant database; the computer uses a deep learning algorithm based on the image acquisition module to extract seed cotyledon phenotypic parameters, extract and repair seed roots, and establish a seed vitality prediction model.

2. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The robotic arm comprises a support plate, a pneumatic guide rail, a pneumatic slider, an axis arm one, an axis arm two, an axis arm three, a robotic arm gripper, a suction nozzle and a motor; a pneumatic guide rail is installed on the top of the support plate, and a pneumatic slider is slidably connected to the side of the pneumatic guide rail; an axis arm one is fixed to the side of the pneumatic slider, and a motor is provided at the end of the axis arm one; the output end of the motor is connected to the axis arm two, and a motor is provided at the end of the axis arm two; the output end of the motor is connected to the axis arm three, and a motor is provided at the end of the axis arm three; the output end of the motor is connected to the robotic arm gripper, and the robotic arm gripper clamps the suction nozzle; the other end of the suction nozzle is connected to a negative pressure vacuum pump.

3. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The water replenishment component includes a water tank, a nozzle, a photoelectric sensor 1, a water pump, a mounting seat, a single-chip microcomputer, a support plate and a relay; a single-chip microcomputer and a relay are provided on the side of the mounting seat, and a support plate is provided at the top of the mounting seat; a water tank is provided on the top of the support plate, and a water pump is provided inside the water tank; the output end of the water pump is connected to the nozzle through a water pipe, and a photoelectric sensor 1 is provided on the side of the support plate; the photoelectric sensor 1, the water pump and the relay are connected to the single-chip microcomputer; the photoelectric sensor 1 and the relay are used to control the switch of the water pump; when the culture dish passes through the photoelectric sensor 1, the photoelectric sensor 1 sends a signal to the single-chip microcomputer for processing, and after receiving the signal, the single-chip microcomputer sets the IN terminal to a high level, controls the relay to be attracted, and then controls the water pump switch.

4. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The camera 1 is used to obtain pictures of the seeds at various angles during rotation, including the robot arm sucking the seeds and rotating them. The camera 1 takes pictures of the seeds rotating one circle. Between each picture, the seeds rotate 15°, for a total rotation of 360°. A single seed requires 24 pictures.

5. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The three-dimensional reconstruction of the seed comprises: Get multi-angle images of the seeds and ensure that the image sizes are consistent; Preprocessing the acquired multi-angle images; Calibrate camera one and perform contour extraction; Perform fixed-point connection and texture mapping for the surface of the 3D model; Perform three-dimensional reconstruction of seeds.

6. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The method for extracting the phenotypic parameters of seed cotyledons includes: using the YOLOv7-OBB recognition algorithm; specifically: YOLOv7-OBB uses a rotated rectangular box as an anchor box; each label contains x, y (horizontal and vertical coordinates of the center of the ellipse), rx, ry (the major and minor axes of the ellipse), and θ (the rotation angle of the ellipse); the ellipse label is converted into a minimum circumscribed rectangular label with a rotation angle; the YOLOv7-OBB network BackBone model is CSPDarknet, which is used to extract features from the input image; the PANet structure is used as an enhanced feature extraction network of the model to fuse features from different levels of CSPDarknet; the feature points are judged by the YOLO Head in the prediction network to generate a detection box and the corresponding target category; the rotation angle is added to the output of the Yolo-Head of YOLOv7-OBB; KLD_Loss is used as the loss function; The images of seeds in different time series are input into the YOLOv7-OBB model in real time. The model recognizes the seeds and outputs the anchor frame parameters, which are sorted and numbered according to the center point coordinates. The rectangular anchor frame is converted into a minimum inscribed ellipse, and the seed elliptical anchor frame is mapped to a blank image of the same size as the original image, where the pixel value of the elliptical anchor frame area is set to 255 and the background pixel value is 0, that is, a single seed binary image. The images of different time series are preprocessed by image enhancement and contrast enhancement, and then the images are converted from RGB to HSV color space, and the color features are used to implement threshold segmentation to obtain a binary image. Combined with the center point coordinates of the anchor frame, the single seed binary image is extracted according to the region growing algorithm. The length, width, aspect ratio, area, perimeter and circularity shape characteristics corresponding to the single seed are detected by contour extraction, which are the phenotypic parameters of the seed in the absorption and expansion state.

7. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The seed root system extraction and root cutting repair comprises: The number of connected regions N is used to determine whether a single seed image has broken roots, and N-2 is used as the number of times to repair broken roots. Under the eight-connected domain, the center point of one adjacent skeleton point is defined as an endpoint, and this point is defined as a common point. Traverse the skeleton graph of the connected area in sequence, extract the endpoint coordinates respectively and sort them by Y-axis coordinates, and extract the minimum and maximum points on the Y-axis; due to the geotropism of the root, the minimum and maximum points on the Y-axis are taken as the root tip and root tail points according to the position characteristics, and the remaining points are the breakpoints; According to the order of the breakpoints, two breakpoints are taken in turn for second-order polynomial fitting until the number of root repairs is reached; The single seed binary image is repaired with the average width of two connected areas; in order to calculate the length and width of the main root, the lateral roots in the binary image are pruned by the lateral root separation algorithm of the contour; unlike the thinned skeleton image, there is no determination of endpoints and intersections in the binary image; the Douglas-Peuker algorithm is used to detect the feature points in the contour and number the feature points counterclockwise; by observing the root growth rule, the lateral root tail is a convex point, and the intersection with the main root is a concave point. The separated lateral roots can be connected according to the two concave points near the convex point of the lateral root; therefore, since the feature point at the root tip must be a convex point, this point is used as the reference point; the convexity of the feature point is judged by the cross product result; if the coordinates of the feature point at the root tip are (x0, y0), and the coordinates of the current feature point are (x1, y1), then the cross product formula is: cross=x0×y1-x1×y0 The sign of the cross product is used to determine whether it is a convex point or a concave point; after determining the convex point, the angle between the convex point and the previous feature point and the next feature point is calculated, and the convex points whose angles are close to the straight line are removed; thus, the number of lateral roots is determined based on the number of convex points; thus, the lateral roots are separated by connecting the previous concave point and the next concave point of the convex point.

8. A seed multi-parameter phenotype detection device according to claim 1, characterized in that: The step of establishing a seed vigor prediction model comprises: The vitality prediction model adopts a long short-term memory (LSTM) model; Adam is used to train the LSTM model, and it is trained in small batches of size 16, with a learning rate of 0.001 and 100 epochs. The LSTM model consists of a 64-neuron LSTM layer and a 32-neuron LSTM layer, and a dropout regularization of 0.2 is applied to prevent the model from overfitting; the LSTM model uses mean square error (MSE), root mean square error (RMSE) and R-squared indicators to evaluate the performance and prediction accuracy of the model.

9. A seed multi-parameter phenotypic detection method, according to any one of claims 1 to 8, characterized in that: include: A mechanical arm is used to suck the seeds and rotate them; Using camera 1 to obtain pictures of seeds at various angles during rotation, and completing three-dimensional reconstruction of the seeds based on the pictures obtained by camera 1, obtaining its morphological characteristic parameters and establishing a relevant database; A chain conveyor is used to transport the culture dish containing the seeds, and a water supply component is used to supply water to the seeds in the culture dish; Camera 2 is used to collect seed images in different time periods according to the changes in the laser ranging module. The deep learning algorithm is used according to the image acquisition module to extract the phenotypic parameters of seed cotyledons, extract and repair the root system of seeds, and establish a seed vitality prediction model.

Images