A non-contact method for correcting external phenotypic parameters and spectra of single soybean leaves

Abstract

The present invention discloses a non-contact method for correcting the external phenotypic parameters and spectrum of a single soybean leaf, comprising: S1, establishing a soybean leaf instance segmentation database; S2, obtaining the RGB information, depth information, and multi-spectral information of the soybean leaf; S3, obtaining a multi-spectral image of a single leaf; and simultaneously obtaining the depth information and multi-spectral information corresponding to each leaf; S4, calculating the external phenotypic parameters of the soybean leaf according to the RGB information and depth information of the single soybean leaf, and using the external phenotypic parameters to correct the spectrum information as the final spectrum correction result. The present invention uses a depth camera to obtain the external parameters of a potted plant, thus avoiding a complicated modeling process, saving time, and reducing the amount of calculation. At the same time, the spectrum correction algorithm can correct the influence of the angle on the spectrum, and no longer requires a handheld instrument to be clamped, which greatly improves the efficiency, and has the effect of improving the efficiency of phenotypic acquisition and breeding.

Description

Technical Field

[0001] The present invention belongs to the field of new generation information technology, especially smart agriculture, and specifically is a contactless soybean single leaf external phenotypic parameter and spectrum correction method. Background Art

[0002] Soybean is native to East Asia and is an important crop. It has strong vitality, wide adaptability, low soil requirements, and can grow under a variety of environmental conditions. As one of my country's main oil crops, soybean not only plays an important role in agricultural production, but is also widely used in food industry, feed production and biofuels. Soybean is rich in protein, fat and various vitamins, and is a high-quality source of people's daily diet. In recent years, with the continuous enhancement of people's health awareness, the nutritional value and functional role of soybean have gradually been recognized by more consumers. The phytoestrogens and dietary fiber contained in soybeans are believed to help lower cholesterol, improve cardiovascular health, and even help regulate weight, so it has become one of the recommended ingredients for a healthy diet. In my country, the planting area of ​​soybeans has expanded year by year, mainly distributed in the northern region, especially in Heilongjiang, Jilin, Liaoning and other places. At present, the soybean industry is developing in the direction of scale and standardization. Many places promote the planting and research and development of high-quality soybeans to enhance market competitiveness. With the popularization of the concept of sustainable development, the demand for biological agriculture and organic soybeans is also increasing, bringing a broader development space for the soybean industry. Overall, soybeans are not only an important agricultural crop, but their diverse application scenarios also bring unlimited possibilities for future development.

[0003] Leaves play a decisive role in the growth and physiological activities of soybeans. As the main organs for plant transpiration, photosynthesis and respiration, leaves are an important place for soybeans to synthesize organic matter required for growth and carry out physiological and biochemical activities. The growth and development status of soybean plants can be observed from the physiological state and health status of the leaves, among which the leaf area index can effectively indicate the health of soybean leaves. Leaf area, that is, the flat area of ​​the leaves, is an important parameter for studying the material circulation and energy conversion between plants and the outside world. It has a significant impact on light energy absorption, water transpiration, organic matter accumulation, yield and economic benefits, and can intuitively reflect the physiological cycle of soybeans.

[0004] In addition, the multispectral imaging technology of leaves can be used to further study the physiological state of soybean leaves. Multispectral imaging technology can obtain detailed information on the health status, nutritional level, water status, etc. of soybean leaves by capturing the reflective spectral information of leaves in different bands. Spectral reflectance in different bands can reveal physiological changes inside leaves, such as nitrogen content, chlorophyll content, and pest and disease conditions. Combining multispectral imaging technology with leaf area indicators can more accurately monitor and evaluate the growth of soybean plants, thereby optimizing planting management and increasing yield and economic benefits.

[0005] At present, there are many methods for leaf area measurement, but there are still certain defects. Traditional basic measurement methods mostly rely on manual operation, which is time-consuming, labor-intensive and costly; and more intelligent measurement techniques often need to be performed with the leaves in vitro, which may cause certain damage to the plants. Therefore, seeking an effective and non-destructive leaf area measurement method has become an important research direction for improving soybean growth monitoring and management.

[0006] Leaf multispectral imaging technology shows great potential in this regard. By capturing the reflectance spectral information of leaves in different bands, the physiological state and leaf area of ​​leaves can be accurately assessed without damaging the plants. However, the application of multispectral imaging technology also faces challenges. One of the main problems is the influence of the spectral acquisition angle on the measurement results. Different incident light angles and observation angles will cause changes in the reflectance spectrum of the leaf surface, thus affecting the accuracy of the measurement. Therefore, seeking a method that can eliminate or reduce the angle effect has become the key to improving the accuracy of multispectral imaging technology.

[0007] The measurement of plant growth status, especially leaf area, urgently needs an efficient, accurate and non-destructive detection method. Manual measurement is not only costly but also accompanied by certain errors. With the development of computer technology, automatic detection technology for plant appearance parameters has begun to appear.

[0008] O'Neal et al. (2002) published an article in the Journal of Economic Entomology that proposed a method for measuring leaf area using a scanner and general software. This method is more efficient than manual work, but because each leaf must be removed from the plant and scanned into a digital format, it is currently very inefficient.

[0009] Igathinathane et al. (2006) published an article in the journal Computers and E-Agriculture, which described the development of an interactive computer software for leaf area measurement. The core of this research was to design and implement a user-friendly software tool to improve the efficiency and accuracy of leaf area measurement. His direction was reasonable, but because the software was a semi-automatic program, the interactive process was complicated and cumbersome, so only certain results were achieved.

[0010] CN 111583328 A disclosed in 2020 a three-dimensional estimation method for the external phenotypic parameters of green radish leaves based on a geometric model. The green radish is photographed once with a fixed RGB-D camera angle and height, and its external phenotypic parameters can be estimated with only one shot. The error range of the estimation result is small and very stable. However, it cannot adapt to plants of different heights at the same time and there is no measurement of leaf inclination.

[0011] In 2020, Hu Peilun et al. selected four types of leaves as research objects and used hyperspectral to measure the reflectivity of the leaves at different incident angles. The ASD spectrometer was then used to measure the reflectivity of the leaves to verify the measurement results of the HSL system. The conclusion is that the hyperspectral measurement results of the leaves are significantly affected by the incident angle. However, they only established a model for the change of leaf reflectivity with the incident angle, and did not conduct in-depth research on the relationship between leaf inclination angle and spectrum.

[0012] In summary, the three-dimensional measurement technology for crop plants is relatively mature, and can achieve a certain speed and accuracy, and can obtain the common appearance parameters of crops. However, in the process of data acquisition, multi-view data collection and registration are required, but changing the position of the plant or the camera is troublesome and consumes manpower and material resources. And there is still a lack of accurate spectral methods for obtaining single leaves. Summary of the invention

[0013] In view of the problems existing in the background technology, the present invention proposes a non-contact method for correcting the external phenotypic parameters and spectrum of a single soybean leaf, which comprises the following steps:

[0014] S1. Establish a soybean leaf instance segmentation database, train an instance segmentation model, and obtain a visual algorithm for the external phenotypic parameters of soybean leaves;

[0015] S2, fixing the RGB-D camera and the multispectral camera at the same height, taking a photo of the soybean potted plant to obtain the soybean RGB image, from which the RGB information, depth information, and multispectral information of the soybean leaves are obtained;

[0016] S3, aligning the RGB information, depth information, and multispectral information of the soybean leaves, and using the visual algorithm of S1 to segment the soybean RGB image obtained by S2 to obtain a single leaf multispectral image; and at the same time, obtaining the depth information and multispectral information corresponding to each leaf;

[0017] S4. Calculate the external phenotypic parameters of the soybean leaf according to the RGB information and depth information of the soybean leaf, and use the external phenotypic parameters to correct the spectral information as the final spectral correction result.

[0018] Specifically, S1 includes the following steps:

[0019] S11, taking photos of the growth status of the selected soybeans;

[0020] S12, annotating the taken photos and constructing a soybean leaf instance segmentation dataset;

[0021] S13, using the data set to divide the training set and the validation set;

[0022] S14. Select YOLOV8 as the instance segmentation model and optimize the model structure;

[0023] S15, using the test set to train the model, and using the validation set to compare the results, if the accuracy is improved, go to S16, otherwise go to S14;

[0024] S16. Use the optimized YOLOV8 as a visual algorithm to obtain external phenotypic parameters of soybean leaves.

[0025] Specifically, S2 includes the following steps:

[0026] S21. Fix the depth camera D435i and the multispectral camera on the supporting device, adjust the height of the camera according to the height of the soybean pot, and keep the distance from the pot at an appropriate distance;

[0027] S22, controlling the depth camera and the multispectral camera to simultaneously shoot the potted plant, so as to obtain RGB photos, depth information and multispectral information at the same time.

[0028] Specifically, S3 includes the following steps:

[0029] S31, registering the depth camera with the multispectral camera;

[0030] S32, using the visual algorithm described in S1 to segment the photographed photos to obtain all single leaf images;

[0031] S33, using a feature point matching algorithm to achieve registration of the single leaf image and the multispectral image, thereby segmenting the single leaf multispectral image.

[0032] Specifically, the feature point matching algorithm includes:

[0033] The descriptor generated by the SIFT algorithm is expressed as: D(p) = [d1, d2, ..., dn], where p is the position of the feature point, D(p) is its descriptor, and n is the dimension of the descriptor;

[0034] The matching of feature points is based on the distance metric between descriptors, and the reliability of the match is judged by the ratio r of the best matching distance to the second best matching distance:

[0035]

[0036] in is the nearest neighbor of p1, is the next nearest neighbor of p1, Indicates the distance between the two;

[0037] After the feature points are matched, the relationship between p1 and p2 is represented by the transformation matrix H: p2 = Hp1.

[0038] Specifically, the acquisition of the leaf external phenotypic parameters includes the following steps:

[0039] S41, calculating the inclination angle of the blade using the depth information: taking out one hundred points from the multispectral image of the single leaf in S3 using a random sampling algorithm, fitting these one hundred points into a plane, and at this time, using this plane as the plane where the blade is located, and directly obtaining the corresponding matching depth information from the depth information according to the points in the RGB image;

[0040] S42, calculating the angle information of the plane using the pixel coordinates of the 100 points and their depth information;

[0041] S43, calculating the leaf external phenotypic parameters using the single leaf multispectral image segmented in S3.

[0042] Specifically, leaf external phenotypic parameters include:

[0043] Leaf area refers to the fitted plane area S of the leaf;

[0044] The blade inclination angle refers to the intersection angle between the blade fitting plane and the three planes of the world coordinate system XOY, XOZ, and YOZ, which are RX, RY, and RZ respectively;

[0045] The leaf shape equation refers to the equation of the ellipse that is closest to the leaf shape.

[0046] Specific:

[0047] 1) The fitted plane area S of the blade is obtained by the following steps:

[0048] Project the three-dimensional points onto the two-dimensional plane XOY to obtain the point set {(xi,yi)}. Using the convex hull algorithm, the convex hull of the two-dimensional point set is composed of (x1,y1),(x2,y2),…,(xn,yn), then the area S of the convex polygon is:

[0049]

[0050] 2) The blade inclination angle θ is obtained by the following steps:

[0051]

[0052] where n1·n2 is the dot product of the vector ||n|| and ||n|| is the magnitude of the vector;

[0053] 3) The leaf shape equation is:

[0054]

[0055] Among them, (h, k) is the coordinate of the center of the ellipse, and cv2.fitEllipse is used to fit the ellipse; a and b are the major and minor axis radii of the ellipse respectively.

[0056] Specifically: The specific steps for correcting spectral information using external phenotypic parameters are:

[0057] 1) Calculate the angle correction coefficient for each pixel, for each pixel (x i ,y i ), whose depth is z i , calculate the angle φ between the pixel and the leaf normal vector i :

[0058]

[0059] Among them, x i ,y i ,z i : represents the position coordinates of the pixel point in the image; a, b, c: these three parameters constitute the leaf surface normal vector n → =(a,b,c), where a represents the component of the normal vector in the x direction, b represents the component of the normal vector in the y direction, and c is usually used to represent the component of the normal vector in the z direction; h,k,z c :These are the coordinates of the reference point on the blade surface, h is the coordinate of the reference point in the x direction, k is the coordinate of the reference point in the y direction, z c is the coordinate of the reference point in the z direction;

[0060] 2) According to φ i Correct the multispectral value of each pixel:

[0061] I′ i =I i ·cos(φ i )

[0062] In the formula, I i Represents the original multispectral value of the pixel, I′ i Represents the corrected pixel multispectral value.

[0063] Beneficial Effects of the Invention

[0064] The present invention uses a depth camera to obtain the external parameters of the potted plants, avoiding the complicated modeling process, saving time and reducing the amount of calculation. At the same time, the spectrum correction algorithm can correct the influence of the angle on the spectrum, and no longer needs to be clamped by a handheld instrument, which greatly improves the efficiency and has the effect of improving the efficiency of phenotypic acquisition and breeding. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 This is a leaf image annotated using labelme

[0066] Figure 2 Schematic diagram of the optimized YOLOV8 structure

[0067] Figure 3 Schematic representation of the instance segmentation model effect comparison

[0068] Figure 4 Schematic diagram of the experimental blade

[0069] Figure 5 Schematic diagram of the leaf shape equation

[0070] Figure 6 Schematic diagram of image segmentation results DETAILED DESCRIPTION

[0071] The present invention will be further described below in conjunction with embodiments, but the protection scope of the present invention is not limited thereto:

[0072] The present invention takes soybean as the research object, uses Intel depth camera and multispectral camera to shoot soybean, obtains RGB information, depth information, and multispectral information of soybean at a single angle, obtains the external phenotypic parameters of a single soybean leaf after processing the RGB information and depth information, and uses some external parameters to correct the spectrum.

[0073] 1.1 Experimental Materials

[0074] The experiment selected several pots of soybeans in good growth state. A total of 2,000 photos of leaves in different positions were taken evenly from top to bottom at an angle perpendicular to the leaves as the research object.

[0075] 1.2 Instance Segmentation Algorithm Selection

[0076] The visual algorithm selected is YOLOv8, which is a deep learning algorithm for real-time object detection, known for its efficient and fast object detection capabilities. YOLOv8 has been improved on the basis of previous versions, further improving the speed and accuracy of detection.

[0077] 1.3 Algorithm Improvement

[0078] like Figure 1As shown in the figure, labelme is used to annotate each leaf in the two thousand photos taken, so as to construct a soybean leaf instance segmentation dataset, and the YOLOV8 model is optimized based on this dataset. The BiFPN module is integrated into the conect in the Head of YOLOV8. BiFPN significantly improves the performance of the feature pyramid network through bidirectional feature fusion and learnable weighting mechanism, and achieves good results in image segmentation. At the same time, the self-attention module and the convolution module are fused with the Yolov8 detection head to form a self-attention SAttention detection head. The self-attention module consists of the following parts: a 1x1 convolution layer to reduce the feature dimension (c1->c2) and reduce the computational load of self-attention, a multi-head self-attention layer (MHSA) to capture the long-distance dependency between features, another 1x1 convolution layer to restore the feature dimension (c2->c1), and a batch normalization layer (BN) for normalization in network training. The addition operation adds the output of the self-attention module to the initial input to form a residual connection and a ReLu activation function. The convolution module consists of the following parts: a downsampling step, which reduces the spatial resolution by 0.5 times downsampling and 3x3 convolution to reduce the amount of calculation, a 1x1 convolution layer for feature conversion, an upsampling step, which restores the spatial resolution by 2 times upsampling, a batch normalization layer (BN), an addition operation, which adds the upsampled output to the input before downsampling, implements skip connections, and a ReLu activation function. This is used to build an efficient structure for high-resolution feature extraction. The structure before and after modification is compared. Figure 2 The optimized model is used to identify soybean potted plants. The results are shown in Figure 3 shown.

[0079] 1.4 Results comparison

[0080]

[0081] Through ablation experiments, it was found that the modified model improved the accuracy, mAP50, and mAP50-9 by 5.1%, 2%, and 2.5% respectively compared with the original model.

[0082] 2.1 Device Introduction:

[0083] The structure of the device of the present invention can be divided into three parts: a chassis, a vertical slide rail, and a camera frame. The chassis is used to place the soybean potted plants to be measured, the vertical slide rail is used to adjust the height of the device, and the camera frame is used to fix the depth camera and the multi-spectral camera.

[0084] 2.2 Depth Camera Calibration:

[0085] Since the calculation results are all in the world coordinate system, it is necessary to convert the object captured by the camera from the pixel coordinate system to the camera coordinate system and then to the world coordinate system. The world coordinate system describes the camera position in meters. The commonly used coordinates are: X W ,Y W ,Z W The origin of the camera coordinate system is located at the light point of the lens. The x and y axes are parallel to the two sides of the image plane, and the z axis is the optical axis of the lens, which is perpendicular to the image plane. Unit: meter, commonly used coordinate representation: X C , Y C ,Z C The origin of the image coordinate system is the midpoint of the imaging plane, unit: millimeter, commonly used coordinates: X, Y. The origin of the pixel coordinate system is the upper left corner of the image, unit: pixel, commonly used coordinates: u, v, which respectively represent the number of columns and rows of the pixel in the image.

[0086] Convert the world coordinate system to the camera coordinate system:

[0087]

[0088] Among them, the rotation matrix R∈R 3×3 , translation matrix t∈R 3×1

[0089]

[0090] Convert the camera coordinate system to the image coordinate system:

[0091] The point (XC, YC, ZC) in the camera coordinates is projected to the point (x, y) in the image coordinates. According to the principle of similar triangles, the proportional relationship between the two points can be obtained:

[0092]

[0093] Therefore, the point (x, y) in the image coordinate system can be expressed as:

[0094]

[0095] So it can be converted into a matrix representation as:

[0096]

[0097] Where f is the focal length.

[0098] 2.3 Obtain RGB information, depth information and multi-light harmonic information of the potted plant to be tested

[0099] Fix the depth camera D435i and the multi-spectral camera on the device, adjust the slide rail according to the height of the soybean pot, and keep the camera height and the distance between the pot at about 300mm. Control the depth camera and the multi-spectral camera to take a photo of the pot at the same time (in the preferred embodiment, this operation can be achieved through software, and the software needs to have the following functions: first, control the depth camera and the multi-spectral camera to take photos to ensure that soybean information is obtained at the same time; second, encapsulate the YOLOV8 and spectral correction algorithm; users can perform the operations they want according to the buttons on the software interface, such as: shooting, segmentation and correction), so as to obtain RGB photos, depth information and multi-spectral information at the same time. Figure 4 Shown is an image of a single soybean leaf used in the experiment.

[0100] 3.1 Obtaining single leaf RGB information and multi-light harmonic information of the potted plant to be tested

[0101] The optimized YOLOV8 model is used to segment the soybean RGB image to obtain the single leaf RGB information of the potted plant to be tested. The feature point matching algorithm is used to match the single leaf RGB information with the corresponding multispectral information, and the single leaf multispectral information is cut out.

[0102] Feature point matching algorithm:

[0103] First, the descriptor generated by the SIFT algorithm can be expressed as: D(p) = [d1, d2, ..., dn],

[0104] Where p is the position of the feature point, D(p) is its descriptor, and n is the dimension of the descriptor. The matching of feature points is based on the similarity measure between descriptors. The distance measure includes the Euclidean distance (or L2 norm), which is defined as:

[0105]

[0106] Here D(p1) and D(p2) are the descriptors of feature points p1 and p2 in images I1 and I2, respectively. For each feature point p1∈P1, find the point p2 whose descriptor is most similar in P2. In order to reduce false matches, the ratio r of the distance of the best match to the distance of the second best match is used to judge the reliability of the match:

[0107]

[0108] Among them, p21 is the nearest neighbor point, and p22 is the next nearest neighbor point. Generally, a threshold such as r<0.8 is taken to ensure the effectiveness of the matching. After matching the feature points, these point pairs can be used to estimate the transformation relationship between the images (such as affine transformation or homography matrix). Assume that the relationship between p1 and p2 can be expressed by the transformation matrix H:

[0109] p2=Hp1

[0110] Here H needs to be calculated through at least 4 pairs of matching points. The least squares method or RANSAC algorithm is used here to further improve the robustness of the matching.

[0111] 4.1. Calculation of leaf external phenotypic parameters

[0112] Fit the plane where the leaf is located according to the three-dimensional coordinates on the acquired single leaf image: Use the RANSAC algorithm to fit the plane, such as Figure 5 As shown in the figure, the RANSAC algorithm randomly selects one hundred points in the leaf area, uses the three-dimensional coordinates of these one hundred points to fit the plane, and obtains the coefficients a, b and intercept c of the plane equation. The normal vector is [a, b, -1]. The plane equation is: z = ax + by + c.

[0113] To calculate the leaf area, first, we project the three-dimensional points onto the two-dimensional plane XOY to obtain the point set {(xi,yi)}. Then, using the convex hull algorithm, assume that the convex hull of the two-dimensional point set consists of points (x1,y1), (x2,y2),…, (xn,yn), then the area A of the convex polygon is calculated as:

[0114]

[0115] In order to calculate the angle between the blade inclination (fitting plane and XOZ, YOZ, XOY plane), the dot product of the normal vector and the cosine theorem are used. The angle θ between two vectors n1 and n2 is calculated as:

[0116]

[0117] where n1·n2 is the dot product of the vectors ||n|| and ||n|| is the magnitude of the vectors.

[0118] To calculate the leaf morphology equation, use cv2.fitEllipse to fit the ellipse. The standard equation of the ellipse is:

[0119]

[0120] Where (h, k) are the coordinates of the center of the ellipse, and a and b are the major and minor radii of the ellipse, respectively.

[0121] Mark the ellipse and the center of the circle on the leaf image, such as Figure 6 shown.

[0122] 4.2 Correction of spectral information

[0123] First, calculate the angle correction coefficient of each pixel, and for each pixel (x i ,y i ), whose depth is z i, calculate the angle φ between the pixel and the leaf normal vector i Pixels (x i ,y i ) and the blade center (h, k) is:

[0124] v i =[x i -h,y i -k,z i -z c ]

[0125] where z c is the depth value of the blade center, then calculate φ i :

[0126]

[0127] Further expansion yields:

[0128] n.v1=a(x1-h)+b(y1-k)-(z1-z e )

[0129]

[0130] then:

[0131]

[0132] According to the calculated angle φ i , using the law of cosines to correct the multispectral value of each pixel:

[0133] I i ′=I i ·cos(φ i )

[0134] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A non-contact method for correcting the external phenotypic parameters and spectrum of a single soybean leaf, characterized in that It includes the following steps: S1. Establish a soybean leaf instance segmentation database, train an instance segmentation model, and obtain a visual algorithm for the external phenotypic parameters of soybean leaves; S2, fixing the RGB-D camera and the multispectral camera at the same height, taking a photo of the soybean potted plant to obtain the soybean RGB image, from which the RGB information, depth information, and multispectral information of the soybean leaves are obtained; S3, aligning the RGB information, depth information, and multispectral information of the soybean leaves, and using the visual algorithm of S1 to segment the soybean RGB image obtained by S2 to obtain a single leaf multispectral image; and at the same time, obtaining the depth information and multispectral information corresponding to each leaf; S4. Calculate the external phenotypic parameters of the soybean leaf according to the RGB information and depth information of the soybean leaf, and use the external phenotypic parameters to correct the spectral information as the final spectral correction result; the specific steps of correcting the spectral information using the external phenotypic parameters are as follows: 1) Calculate the angle correction coefficient for each pixel, for each pixel (x i ,y i ), whose depth is z i , calculate the angle φ between the pixel and the leaf normal vector i : Among them, x i ,y i ,z i : represents the position coordinates of the pixel point in the image; a, b, c: these three parameters constitute the leaf surface normal vector n→=(a, b, c), a represents the component of the normal vector in the x direction, b represents the component of the normal vector in the y direction, and c is usually used to represent the component of the normal vector in the z direction; h, k, z c :These are the coordinates of the reference point on the blade surface, h is the coordinate of the reference point in the x direction, k is the coordinate of the reference point in the y direction, z c is the coordinate of the reference point in the z direction; 2) According to φ i Correct the multispectral value of each pixel: I i ′=I i ·cos(φ i ) In the formula, I i Represents the original multispectral value of the pixel, I i ′ represents the corrected pixel multispectral value.

2. The method according to claim 1, characterized in that The S1 comprises the following steps: S11, taking photos of the growth status of the selected soybeans; S12, annotating the taken photos and constructing a soybean leaf instance segmentation dataset; S13, using the data set to divide the training set and the validation set; S14. Select YOLOV8 as the instance segmentation model and optimize the model structure; S15, train the model using the training set, and compare the results using the validation set. If the accuracy is improved, go to S16, otherwise go to S14; S16. Use the optimized YOLOV8 as a visual algorithm to obtain external phenotypic parameters of soybean leaves.

3. The method according to claim 1, characterized in that The S2 comprises the following steps: S21. Fix the depth camera D435i and the multispectral camera on the supporting device, adjust the height of the camera according to the height of the soybean pot, and keep the distance from the pot at an appropriate distance; S22, controlling the depth camera and the multispectral camera to simultaneously shoot the potted plant, so as to obtain RGB photos, depth information and multispectral information at the same time.

4. The method according to claim 1, characterized in that: The S3 comprises the following steps: S31, registering the depth camera with the multispectral camera; S32, using the visual algorithm described in S1 to segment the photographed photos to obtain all single leaf images; S33, using a feature point matching algorithm to achieve registration of the single leaf image and the multispectral image, thereby segmenting the single leaf multispectral image.

5. The method according to claim 4, characterized in that The feature point matching algorithm includes: The descriptor generated by the SIFT algorithm is expressed as: D(p) = [d1, d2, ..., dn], where p is the position of the feature point, D(p) is its descriptor, and n is the dimension of the descriptor; The matching of feature points is based on the distance metric between descriptors, and the reliability of the match is judged by the ratio r of the best matching distance to the second best matching distance: where p 21 is the nearest neighbor of p1, p 22 is the next nearest neighbor of p1, d(p1,p 21 ) represents the distance between the two; After the feature points are matched, the relationship between p1 and p2 is represented by the transformation matrix H: p2 = Hp1.

6. The method according to claim 1, characterized in that The acquisition of the leaf external phenotypic parameters comprises the following steps: S41, calculating the inclination angle of the blade using the depth information: taking out one hundred points from the multispectral image of the single leaf in S3 using a random sampling algorithm, fitting these one hundred points into a plane, and at this time, using this plane as the plane where the blade is located, and directly obtaining the corresponding matching depth information from the depth information according to the points in the RGB image; S42, calculating the angle information of the plane using the pixel coordinates of the 100 points and their depth information; S43, calculating the leaf external phenotypic parameters using the single leaf multispectral image segmented in S3.

7. The method according to claim 6, characterized in that Leaf external phenotyping parameters include: Leaf area refers to the fitted plane area S of the leaf; The blade inclination angle refers to the intersection angle between the blade fitting plane and the three planes of the world coordinate system XOY, XOZ, and YOZ, which are RX, RY, and RZ respectively; The leaf shape equation refers to the equation of the ellipse that is closest to the leaf shape.

8. The method according to claim 7, characterized in that: 1) The fitted plane area S of the blade is obtained by the following steps: Project the three-dimensional points onto the two-dimensional plane XOY and obtain the point set {(x i, y i )}, using the convex hull algorithm, the convex hull of a two-dimensional point set is (x1,y1),(x2,y2),…,(x n ,y n ), then the area S of the convex polygon is: 2) The blade inclination angle θ is obtained by the following steps: Where n1·n2 is the dot product of the vectors, ||n1|| and ||n2|| are the magnitudes of the vectors; 3) The leaf shape equation is: Among them, (h, k) is the coordinate of the center of the ellipse, and cv2.fitEllipse is used to fit the ellipse; a and b are the major and minor axis radii of the ellipse respectively.

Images