A three-dimensional phenotypic analysis robot and analysis method for wheat in the field

Abstract

The invention relates to a three-dimensional phenotypic analysis robot and an analysis method for field wheat, and relates to the technical field of agricultural auxiliary breeding. The robot realizes the refined analysis of single-plant wheat phenotypic data in a field environment. The robot comprises a walking trolley and a three-dimensional phenotypic analysis device. The walking trolley realizes walking and turning. The three-dimensional phenotypic analysis device can complete the isolation of wheat plants, create a dark environment, and reduce the influence of external environmental factors. The data acquisition device rotates one circle to acquire data and complete the transmission. The collected photos are reconstructed into a sparse point cloud by using an improved colmap, and then densely reconstructed by using an improved mvs algorithm. After removing the background point cloud and reducing noise, a complete and smooth three-dimensional point cloud is obtained. Then, wheat ears, stems, leaves and non-plant point clouds are automatically segmented and wheat plant phenotypic parameters are extracted. The whole process realizes the automatic analysis of wheat phenotypes and improves the accuracy level of single-plant wheat phenotypic analysis in the field.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional data acquisition and phenotypic analysis of wheat, and specifically to a three-dimensional phenotypic analysis robot and analysis method applied to field wheat. Background Art

[0002] Wheat is one of the most important food crops in the world and is vital to global food security. 35%-40% of the world's population uses wheat as a staple food. With the development of genotype identification technology, efficient collection and analysis of crop phenotypes has become a bottleneck in genetic research. Currently, wheat phenotypic data is mostly obtained using traditional methods (manual measurement, visual identification, etc.), which takes a long time to obtain data.

[0003] For wheat plants, geometric parameter information such as leaf area, tiller angle, plant height, tiller number, spike shape and canopy area is crucial for wheat breeding. Automatic extraction of wheat phenotypic characteristics and parameters based on three-dimensional point clouds is a hot topic in current agricultural information technology research. The key to obtaining wheat organ phenotypic information lies in completing the acquisition of wheat three-dimensional data and realizing high-throughput crop phenotypic acquisition, crop plant type characteristic evaluation, and plant structure and phenotypic correlation analysis based on three-dimensional phenotypic analysis methods. Due to the dense growth of wheat, it is difficult to achieve fine acquisition and analysis of single wheat phenotypes from field planting. At the same time, the current method based on three-dimensional scanners has a relatively fast data acquisition speed, no damage to plants, and a large amount of three-dimensional point cloud data, but it is relatively time-consuming to process and the equipment is expensive. At the same time, this method has high requirements for the environment, especially metal objects are not allowed around during scanning. The method based on depth camera obtains a depth map through a camera, and fuses the RGB image with the depth map to obtain point cloud information. The speed is fast, but this method is prone to generate a large number of noisy point clouds, and requires the registration of multiple viewpoint point clouds, which is cumbersome to process and has low accuracy. Therefore, it is of great significance to carry out relevant research on wheat three-dimensional phenotyping robot. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a three-dimensional phenotypic analysis robot and analysis method for field wheat, which solves the problem that three-dimensional data of field wheat is difficult to obtain and the problem that the accuracy of three-dimensional phenotypic analysis results of crops in the prior art is low.

[0005] The present invention is achieved through the following technical scheme, which provides a three-dimensional phenotyping robot for field wheat, including a walking trolley and a three-dimensional phenotyping device, the walking trolley is equipped with a third industrial camera and a double guide rail mechanism for driving the three-dimensional phenotyping device to move horizontally and vertically, the three-dimensional phenotyping device includes a fixed shell and a plurality of circumferentially arranged supporting links, one end of the supporting link is hinged to the fixed shell, and the other end is hinged to a lever, and flexible shading sheets are installed between adjacent supporting links and between adjacent levers, the three-dimensional phenotyping device also includes an expansion driving mechanism for driving the supporting links and the lever to swing outward, the fixed shell is equipped with a first industrial camera located in a space surrounded by a plurality of supporting links, and the space surrounded by a plurality of supporting links is also equipped with a data acquisition device, the data acquisition device includes a circumferentially moving track trolley and a second industrial camera installed on the track trolley.

[0006] The walking trolley in this scheme can realize walking and turning. After the supporting connecting rod and the lever are swung outward and unfolded, the wheat plants are isolated by the flexible shading sheet to create a dark environment and reduce the influence of external environmental factors. The wheat is photographed downward by the first industrial camera, and the track trolley moves circumferentially, and the second industrial camera is used to take pictures of the wheat circumference, thereby realizing the three-dimensional phenotypic analysis of wheat.

[0007] As an optimization, the walking trolley includes a frame and legs installed at the four corners of the frame, and the lower ends of the legs are connected to a walking mechanism through a steering device. The walking mechanism in this solution realizes walking, and the steering device realizes steering.

[0008] As an optimization, the legs are connected to the frame through a vehicle body adjustment device, which includes a width adjustment shell fixed to the frame and an extrusion plate laterally slidably connected to the width adjustment shell, the legs are fixed to the extrusion plate, one end of the width adjustment shell is threadedly connected to an adjustment rod that presses against one side of the extrusion plate, and the other end of the width adjustment shell is equipped with a compression spring that presses against the other side of the extrusion plate. In this solution, the extrusion plate is driven to move left and right by rotating the adjustment rod, thereby adjusting the left and right position of the legs, realizing the width adjustment of the vehicle body in the lateral direction, and adapting to different planting modes.

[0009] As an optimization, the dual-rail mechanism includes an X-transverse rail and a Y-longitudinal rail vertically slidably connected to the slider of the X-transverse rail, and the three-dimensional phenotype analysis device is fixedly connected to the lower end of the Y-longitudinal rail. In this solution, the slider on the X-transverse rail moves left and right, and the Y-longitudinal rail moves up and down on the slider on the X-transverse rail, thereby realizing the horizontal and vertical movement of the three-dimensional phenotype analysis device.

[0010] As an optimization, the deployment drive mechanism includes a folding sleeve rod connected between the support link and the lever, the folding sleeve rod includes a first folding rod and a second folding rod, one end of the first folding rod is hinged to the lever, the other end of the first folding rod is hinged to one end of the second folding rod, the other end of the second folding rod is hinged to the support link, a traction rope is connected to the second folding rod, and the deployment drive mechanism also includes a rope pulling mechanism that drives all traction ropes to pull up synchronously. In this solution, the support link and the lever are swung outwards for deployment by pulling the traction rope upwards.

[0011] As an optimization, the rope pulling mechanism includes a plurality of bevel gear shafts arranged circumferentially and a third motor driving any bevel gear shaft to rotate, and the ends of adjacent bevel gear shafts are connected by bevel gear transmission, and the traction rope is wound around the bevel gear shaft. Through the bevel gear transmission between adjacent bevel gear shafts, the synchronous rotation of all bevel gear shafts is achieved, so that the traction rope is wound around the bevel gear shaft, and the synchronous pulling up of the traction rope is achieved.

[0012] As an optimization, a downward hydraulic rod is fixed in the fixed housing, a slide is fixed at the lower end of the hydraulic rod, and the first industrial camera is mounted on the slide. The hydraulic rod in this solution drives the first industrial camera to move up and down, and retracts upward when not taking pictures, thereby protecting the first industrial camera.

[0013] As an optimization, a circular electric soft rail is installed in the space surrounded by multiple supporting links, the inner side of the supporting links is connected to the electric soft rail through a track fixing frame, and the track trolley moves on the electric soft rail. The electric soft rail in this solution has a certain flexibility. When the supporting links are unfolded, the electric soft rail is circular. When the supporting links are retracted, the electric soft rails between adjacent supporting links are bent and folded.

[0014] As an optimization, a lighting lamp is installed inside the supporting connecting rod. The lighting lamp in this solution plays a supplementary lighting effect.

[0015] A method for analyzing three-dimensional phenotyping of wheat in the field:

[0016] The data acquisition device stops rotating after each rotation of 25 degrees, takes photos with the second industrial camera, and continues to rotate. After one rotation, the acquired photos and the photos collected by the first industrial camera are transmitted to the edge computing device, and the point cloud is reconstructed using the improved colmap algorithm and the improved mvs algorithm. After processing using the through-filter algorithm based on the limit value and the bilateral filtering algorithm based on the k-neighborhood, a complete and smooth three-dimensional point cloud is obtained.

[0017] The improved colmap algorithm reconstructs the sparse point cloud of crops. The sparse point cloud reconstruction content is as follows: first extract the photo features, search for corresponding points, and extract the camera internal parameters from EXIF; then match the extracted features, and generate a scene graph and a matching matrix at the end of the process; then perform incremental modeling, gradually increase the viewing angle, and iteratively optimize the reprojection error to obtain the sparse point cloud of the scene and the camera posture of each viewing angle; finally remove the image distortion, and use optical consistency and geometric consistency to jointly match the bound structure value;

[0018] The improved MVS algorithm reconstructs the dense point cloud of crops. The dense point cloud reconstruction content is as follows: first, the scene depth is estimated to obtain the depth value and normal vector value of the estimated perspective, and the depth optimization is performed using consistency; then the depth map is fused for dense reconstruction; then a coarse grid is established and the grid is refined; finally, the grid texture is mapped to obtain a relatively smooth dense point cloud;

[0019] The through-filter algorithm based on the limit value removes the background point cloud: by setting thresholds in the three directions of X, Y, and Z respectively to enclose the point cloud cube, the target point cloud is enclosed in the bounding box, and the point cloud outside the threshold is deleted;

[0020] The k-neighborhood-based bilateral filtering algorithm removes noise and performs smoothing. First, a kd tree is constructed for the point cloud to describe the location and existence attributes of the points, and then a k-neighborhood analysis is performed. Finally, bilateral filtering is used to remove noise points and the point cloud is smoothed to obtain a complete and smooth three-dimensional crop point cloud. Finally, the wheat ears, stems, leaves and non-plant point clouds are automatically segmented and the phenotypic parameters of the wheat plants are extracted.

[0021] The beneficial effects of the present invention are as follows: a three-dimensional phenotypic analysis robot and analysis method for field wheat of the present invention realizes automatic field straightening through the field wheat three-dimensional data phenotypic analysis robot, realizes automatic collection of wheat three-dimensional data through the three-dimensional phenotypic analysis device; and realizes rapid reconstruction of complete and smooth three-dimensional point clouds through the phenotypic analysis method, thereby realizing lossless reconstruction of wheat three-dimensional phenotypes. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the structure of the present invention;

[0023] Figure 2 It is a structural schematic diagram of another angle of the present invention;

[0024] Figure 3 It is a structural schematic diagram of the vehicle body adjustment device of the present invention;

[0025] Figure 4 It is a schematic diagram of the structure of the extrusion plate and the adjustment rod in the vehicle body adjustment device of the present invention;

[0026] Figure 5 It is a schematic diagram of the internal structure of the vehicle body adjustment device of the present invention;

[0027] Figure 6 It is a schematic diagram of the structure of the three-dimensional phenotypic analysis device of the present invention;

[0028] Figure 7 It is a schematic diagram of the structure of the three-dimensional phenotype analysis device of the present invention in an expanded state;

[0029] Figure 8 It is a top view of the rope pulling mechanism in the three-dimensional phenotype analysis device of the present invention;

[0030] Fig. 9 It is a schematic diagram of the internal structure of the three-dimensional phenotypic analysis device of the present invention in an expanded state;

[0031] Fig.10 It is a schematic diagram of the internal structure of the front side of the three-dimensional phenotyping device of the present invention;

[0032] Fig.11 It is a structural schematic diagram of the position of the hydraulic rod in the three-dimensional phenotyping device of the present invention;

[0033] Fig.12 It is a structural schematic diagram of the position of the folding sleeve rod in the three-dimensional phenotype analysis device of the present invention;

[0034] Fig.13 It is a schematic diagram of the structure of the inner side of the supporting connecting rod in the three-dimensional phenotype analysis device of the present invention;

[0035] Fig.14 It is a schematic diagram of the structure of the data acquisition device in the three-dimensional phenotype analysis device of the present invention;

[0036] Fig.15 It is a structural schematic diagram of a data acquisition device and an electrically flexible rail in a three-dimensional phenotypic analysis device of the present invention;

[0037] Fig.16 It is a structural schematic diagram of the fourth motor position in the three-dimensional phenotype analysis device of the present invention;

[0038] Fig.17 It is a schematic diagram of the three-dimensional data phenotype analysis method of the present invention;

[0039] As shown in the figure:

[0040] 1. Walking mechanism, 2. Steering device, 3. Three-dimensional phenotype analysis device, 31. Fixed shell, 311. Pulley fixing frame, 312. Third bevel gear shaft, 313. Shaft fixing support, 314. Fourth bevel gear shaft, 315. First bevel gear shaft, 316. Second bevel gear shaft, 317. Vertical guide rail, 32. Traction rope, 33. Drag rod, 34. Folding sleeve rod, 341. Second limit baffle, 342. Elastic baffle, 343. Third limit baffle, 344. First folding rod, 345. Second folding rod, 346. Connecting pulley, 347. Spring, 348. Illuminating lamp, 35. Support connecting rod, 36. Electric soft rail, 361. Track fixing frame, 37. Slide, 371. Chassis, 372. First industrial camera, 373. Camera base, 374. Hydraulic rod, 37 5. Fixed seat, 376. First limit baffle, 38. Support pulley, 39. Third motor, 391. Motor fixed base, 392. Coupling, 4. Frame, 41. Triangular support plate, 5. Vehicle body adjustment device, 51. Width adjustment shell, 52. Adjustment rod, 53. Limit slot, 54. Extrusion plate, 55. Compression spring, 6. Third industrial camera, 7. Double guide rail mechanism, 71. X-transverse guide rail, 72. First motor, 73. Y-longitudinal guide rail, 74. Second motor, 75. Fixed base, 8. Edge computing device, 81. Power supply, 9. Data acquisition device, 91. Track trolley, 911. Trolley shell, 912. Belt, 913. Fourth motor, 914. Load-bearing wheel, 915. Synchronous wheel, 92. Connecting rod, 93. Bolt fastener, 94. Second industrial camera. DETAILED DESCRIPTION

[0041] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation method.

[0042] like Figure 1 to Figure 17 As shown, a three-dimensional phenotyping robot for wheat in the field of the present invention comprises a walking vehicle and a three-dimensional phenotyping device 3.

[0043] The walking trolley includes a frame 4 and legs installed at the four corners of the frame 4. The frame 4 is welded and fixed by multiple aluminum alloy rectangular brackets. A third industrial camera 6 is installed at the front end of the frame 4.

[0044] The lower end of the outrigger is connected to a walking mechanism 1 through a steering device 2. The walking mechanism consists of a wheel frame and a hub motor wheel. The walking mechanism 1 is connected to the steering device 2 through an axis. A motor is provided inside the steering device 2. Different motors are controlled to achieve steering of different hub motor wheels, and the maximum steering angle reaches 90 degrees.

[0045] like Figure 2As shown, the two left legs are connected by a cross beam, and the two right legs are also connected by a cross beam. A triangular support plate 41 is welded between the cross beam and the legs to play a reinforcing role.

[0046] like Figure 3-Figure 5 As shown, the support leg is connected to the frame 4 through a body adjusting device 5, and the body adjusting device 5 includes a width adjusting shell 51 fixed to the frame 4 and an extrusion plate 54 laterally slidably connected to the width adjusting shell 51. The width adjusting shell 51 is welded to the frame, and the support leg is inserted into the width adjusting shell 51 from the lower end. A limiting slot 53 is provided at the bottom of the width adjusting shell 51 to limit the left and right adjustment width of the support leg.

[0047] The legs are fixedly connected to the extrusion plate 54, and one end of the width adjustment housing 51 is threadedly connected with an adjustment rod 52 that presses against one side of the extrusion plate 54, and the other end of the width adjustment housing 51 is equipped with a compression spring 55 that presses against the other side of the extrusion plate 54. By controlling the adjustment rod 52, the extrusion plate 54 is driven to move left and right, thereby realizing the left and right lateral adjustment of the legs, and thus realizing the left and right position adjustment of the steering device 2 and the walking mechanism 1. A scale bar is left on the outside of the width adjustment housing 51 for precise adjustment. The width of the vehicle body in the lateral direction is adjusted by the vehicle body adjustment device 5, so as to adapt to different wheat planting modes.

[0048] The walking carriage is provided with a double rail mechanism 7 for driving the three-dimensional phenotype analysis device 3 to move horizontally and vertically. The double rail mechanism 7 includes an X-transverse rail 71 and a second motor 74. The X-transverse rail 71 is fixed on the vehicle frame 4. The second motor 74 drives the slider of the X-transverse rail 71 to move horizontally left and right. The double rail mechanism 7 also includes a Y-longitudinal rail 73 vertically slidably connected to the slider of the X-transverse rail 71 and a first motor 72. The first motor 72 drives the Y-longitudinal rail 73 to move vertically up and down. The Y-longitudinal rail 73 is fixedly mounted with the three-dimensional phenotype analysis device 3 through a fixed base 75.

[0049] The three-dimensional phenotyping analysis device 3 includes a fixed shell 31 and a plurality of circumferentially arranged supporting rods 35. The fixed shell 31 is fixedly connected to a fixed base 75. The fixed shell 31 in this embodiment is cylindrical. An edge computing device 8 is installed in the fixed shell 31. A power supply 81 is installed next to the edge computing device 8. The overall operation control of the field wheat three-dimensional data phenotyping analysis robot is realized through the edge computing device 8.

[0050] The upper end of the support link 35 is hinged to the fixed shell 31, and the lower end is hinged with a lever 33, and the hinge axes are all arranged horizontally, so that the support link 35 can swing outward, and the lever 33 can swing outward relative to the support link 35. The support link 35 and the lever 33 have an initial opening angle in the initial state to facilitate surrounding the collected wheat plants.

[0051] In this embodiment, the support link 35 is hinged to the fixed shell 31 and the first limit baffle 376 is used to limit its outward swinging angle. An elastic baffle 342 and a spring 347 are connected between the support link 35 and the lever 33. The rebound of the spring 347 is used to restore the support link 35 and the lever 33 to their initial state. The bottom (that is, the inner side) of the lever 33 is made of rubber material to avoid causing damage to wheat.

[0052] Flexible shading sheets are installed between adjacent supporting links 35 and between adjacent levers 33. The flexible shading sheets can be made of black cloth or flexible materials such as dark plastic sheets, so as to better isolate the acquired wheat plants from other plant types, while creating a dark environment and reducing the impact of external light.

[0053] A lighting lamp 348 is installed inside the supporting connecting rod 35. The lighting lamp 348 can be a halogen lamp or an LED lamp, which is used to supplement the light source, reduce the influence of external environmental factors, and improve the quality of the acquired data.

[0054] The three-dimensional phenotype analysis device 3 also includes an expansion drive mechanism for driving the supporting link 35 and the lever 33 to swing outwards, such as Fig.12 As shown, the unfolding drive mechanism includes a folding sleeve rod 34 connected between the supporting link 35 and the lever 33, the folding sleeve rod 34 is located on the outside of the supporting link 35 and the lever 33, the folding sleeve rod 34 includes a first folding rod 344 and a second folding rod 345, one end of the first folding rod 344 is hinged to the lever 33 and its rotation angle is limited by the third limit baffle 343, the other end of the first folding rod 344 is hinged to one end of the second folding rod 345, the other end of the second folding rod 345 is hinged to the supporting link 35 and its rotation angle is limited by the second limit baffle 341.

[0055] The second folding rod 345 is connected to a traction rope 32. Specifically, the second folding rod 345 is provided with a connecting pulley 346, and the connecting pulley 346 is connected to the traction rope 32. The unfolding driving mechanism further includes a rope pulling mechanism for driving all traction ropes 32 to be pulled up synchronously. The folding sleeve rod 34 is controlled by driving the retracted and extended length of the traction rope 32 so as to adjust the outward movement of the supporting link 35 and the lever 33. The opening and closing at different angles are achieved by controlling the traction rope 32.

[0056] In this embodiment, a pulley fixing frame 311 is fixed on the upper side of the fixed housing 31, and the pulley fixing frame 311 is connected to the support pulley 38 through an axis to support the traction rope 32 for its steering.

[0057] The rope pulling mechanism includes a plurality of bevel gear shafts arranged circumferentially and a third motor 39 for driving any bevel gear shaft to rotate. The ends of adjacent bevel gear shafts are connected by bevel gear transmission, and the traction rope 32 is wound around the bevel gear shaft. Specifically, Figure 8 As shown, four bevel gear shafts are provided in this embodiment, namely, the first bevel gear shaft 315, the second bevel gear shaft 316, the third bevel gear shaft 312 and the fourth bevel gear shaft 314. A shaft fixing support 313 is fixed inside the upper side of the fixed housing 31, and the shaft fixing support 313 supports the first bevel gear shaft 315, the second bevel gear shaft 316, the third bevel gear shaft 312 and the fourth bevel gear shaft 314 through a bearing. The first bevel gear shaft 315 is connected to the third motor 39 through a coupling 392, and the third motor 39 is installed on the fixed housing through a motor fixing base 391. The third motor 39 drives the first bevel gear shaft 315 to operate, thereby driving the second bevel gear shaft 316, the third bevel gear shaft 312 and the fourth bevel gear shaft 314 to operate at the same speed and in the same outward (inward) direction, thereby driving the retracting and releasing length of the traction rope 32. The traction rope 32 is connected to the bevel gear shaft and bypasses the support pulley 38, and finally realizes the rotation of the support connecting rod 35 and the shifting rod 33 through the traction rope 32.

[0058] The fixed shell 31 contains a first industrial camera 372 located in a space surrounded by a plurality of supporting links 35. The first industrial camera 372 faces downward, and a downward-facing hydraulic rod 374 is fixedly connected to the fixed shell 31. In this embodiment, a fixing seat 375 is fixedly installed at the bottom of the fixed shell 31 for carrying the hydraulic rod 374.

[0059] The lower end of the hydraulic rod 374 is fixedly connected to a slide 37, and the first industrial camera 372 is mounted on the slide 37. The interior of the fixed housing 31 is connected to the slide 37 via a vertical guide rail 317, and the slide 37 is driven up and down by the hydraulic rod 374, and the slide 37 can be raised and lowered in the vertical direction along the vertical guide rail 317. A chassis 371 is fixedly installed inside the slide 37, and a camera base 373 is fixedly installed on the chassis 371. A limiting hole is left on the camera base 373 for fixing and adjusting the position of the first industrial camera 372. When the data analysis is completed, the slide 37 rises to protect the first industrial camera 372.

[0060] A ring-shaped electrified soft rail 36 is installed in the space enclosed by multiple supporting links 35. The inner side of the supporting links 35 is connected to the electrified soft rail 36 through a track fixing frame 361. The electrified soft rail 36 has a certain flexibility. The electrified soft rail 36 is unfolded with the rotation of the supporting links 35. When the supporting links are unfolded, the electrified soft rail is circular. When the supporting links are folded, the electrified soft rails between adjacent supporting links are bent and folded. The electrified soft rail 36 is directly powered by connecting to the data acquisition device 9.

[0061] The space enclosed by the multiple supporting links 35 is also equipped with a data acquisition device 9, which includes a circumferentially movable track trolley 91 and a second industrial camera 94 mounted on the track trolley 91. The track trolley 91 moves on the electric soft rail 36. Fig.14 As shown, the rail trolley 91 and the second industrial camera 94 are connected through a connecting rod 92 and a bolt fastener 93, and the shooting angle of the second industrial camera 94 is adjusted by adjusting the bolt fastener 93 and the connecting rod 92. The rail trolley is composed of a trolley housing 911, a load-bearing wheel 914, a synchronous wheel 915 and a fourth motor 913. The fourth motor is connected to the load-bearing wheel 914 through a belt 912. The rail trolley 91 is driven by the fourth motor 913, driving the data acquisition device 9 to move in a circular direction along the electric soft rail 36.

[0062] A method for analyzing three-dimensional phenotyping of wheat in the field:

[0063] The data acquisition device stops rotating after each rotation of 25 degrees, and the second industrial camera 94 takes photos, and continues to rotate. After one rotation, the acquired photos and the photos acquired by the first industrial camera 372 are transmitted to the edge computing device, and the point cloud is reconstructed using the improved colmap algorithm and the improved mvs algorithm, and then processed using the through-filter algorithm based on the limit value and the bilateral filtering algorithm based on the k-neighborhood to obtain a complete and smooth three-dimensional point cloud;

[0064] The improved colmap algorithm reconstructs the sparse point cloud of crops. The sparse point cloud reconstruction content is as follows: first extract the photo features, search for corresponding points, and extract the camera internal parameters from EXIF; then match the extracted features, and generate a scene graph and a matching matrix at the end of the process; then perform incremental modeling, gradually increase the viewing angle, and iteratively optimize the reprojection error to obtain the sparse point cloud of the scene and the camera posture of each viewing angle; finally remove the image distortion, and use optical consistency and geometric consistency to jointly match the bound structure value;

[0065] The improved MVS algorithm reconstructs the dense point cloud of crops. The dense point cloud reconstruction content is as follows: first, the scene depth is estimated to obtain the depth value and normal vector value of the estimated perspective, and the depth optimization is performed using consistency; then the depth map is fused for dense reconstruction; then a coarse grid is established and the grid is refined; finally, the grid texture is mapped to obtain a relatively smooth dense point cloud;

[0066] The through-filter algorithm based on the limit value removes the background point cloud: by setting thresholds in the three directions of X, Y, and Z respectively to enclose the point cloud cube, the target point cloud is enclosed in the bounding box, and the point cloud outside the threshold is deleted;

[0067] The k-neighborhood-based bilateral filtering algorithm removes noise and performs smoothing. First, a kd tree is constructed for the point cloud to describe the location and existence attributes of the points, and then a k-neighborhood analysis is performed. Finally, the noise points are removed by using bilateral filtering, and the point cloud is smoothed to obtain a complete and smooth three-dimensional point cloud of the crop. Finally, the wheat ears, stems, leaves and non-plant point clouds are automatically segmented and the phenotypic parameters of the wheat plant (plant height, ear length, leaf length, leaf area, leaf curvature, ear width, ear volume and flag leaf inclination) are extracted.

[0068] The action process of the 3D phenotyping robot when performing 3D phenotyping of wheat in the field:

[0069] S1: The third industrial camera 6 takes a photo and sends the photo to the edge computing device 8. The edge computing device 8 controls the wheel hub motor to operate so that the wheat three-dimensional data phenotype analysis robot moves along the wheat ridge into the shooting area and controls the lighting 348 to turn on.

[0070] S2: The edge computing device 8 transmits a signal to the second motor 74, driving the slider of the X-transverse guide rail 71 to translate horizontally, and the first industrial camera 372 collects data and calibrates the position. When the three-dimensional phenotyping device moves directly above the wheat, the second motor 74 stops running. The first motor 72 drives the Y-longitudinal guide rail 73 to translate vertically, and after reaching the set displacement, the information is transmitted to the edge computing device 8.

[0071] S3: The edge computing device 8 transmits a signal to the third motor 39, and the third motor 39 drives the first bevel gear shaft to drive the second bevel gear shaft, the third bevel gear shaft and the fourth bevel gear shaft to rotate synchronously and inward, thereby driving the traction rope to contract. The tension of the traction rope is used to bend the folding sleeve rod to drive the supporting connecting rod and the lever to move outward until the specified angle. A flexible shading sheet is used to connect two adjacent supporting connecting rods and two levers to create a dark environment and, together with the lever, isolate other wheat plants, so that the selected wheat plants are covered inside the three-dimensional phenotyping analysis device.

[0072] S4: After the wheat isolation is completed, the edge computing device transmits a signal to the data acquisition device 9. The data acquisition device 9 stops rotating after each rotation of 25 degrees. The data acquisition device takes a photo and continues to rotate. After one rotation, the data acquisition device returns to the initial position. The edge computing device transmits a signal to the third motor 39 to drive the traction rope to relax. At the same time, under the action of the spring rebound, the support link and the lever return to their initial state. The support link and the lever are not completely retracted to the minimum circumference to avoid incomplete data collection. Finally, the hydraulic rod 374 contracts, the slide moves up to protect the first industrial camera 372, and the first motor 72 drives the Y-longitudinal guide rail 73 to translate vertically.

[0073] S5: After step S4 is completed, the first industrial camera 372 and the data acquisition device 9 transmit the photos to the edge computing device, reconstruct the point cloud using the proposed three-dimensional reconstruction algorithm, and then obtain a complete and smooth three-dimensional point cloud after processing using other algorithms.

[0074] S6: Then, the wheat ears, stems, leaves and non-plant point clouds are automatically segmented from the complete smooth three-dimensional point cloud and the wheat plant phenotypic parameters (plant height, ear length, leaf length, leaf area, leaf curvature, ear width, ear volume and flag leaf inclination) are extracted.

[0075] S7: Repeat steps S2, S3, S4, S5, and S6 to complete multiple three-dimensional phenotypic analyses of individual wheat plants in the field.

[0076] Of course, the above description is not limited to the above examples. Technical features not described in the present invention can be achieved by or by using existing technologies, which will not be described here. The above embodiments and drawings are only used to illustrate the technical scheme of the present invention and are not limitations of the present invention. The present invention is described in detail with reference to the preferred implementation methods. Ordinary technicians in this field should understand that changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention do not depart from the purpose of the present invention, and should also fall within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional phenotyping robot for wheat in the field, characterized in that: The invention comprises a walking trolley and a three-dimensional phenotype analysis device (3), wherein the walking trolley is provided with a third industrial camera (6) and a double guide rail mechanism (7) for driving the three-dimensional phenotype analysis device (3) to move horizontally and vertically, wherein the three-dimensional phenotype analysis device (3) comprises a fixed shell (31) and a plurality of supporting connecting rods (35) arranged circumferentially, wherein one end of the supporting connecting rod (35) is hinged to the fixed shell (31), and the other end is hinged to a lever (33), and flexible connecting rods are provided between adjacent supporting connecting rods (35) and adjacent levers (33). The three-dimensional phenotype analysis device (3) further comprises an unfolding drive mechanism for driving the supporting connecting rod (35) and the lever (33) to swing outwards, the fixed housing (31) is provided with a first industrial camera (372) located in a space surrounded by the plurality of supporting connecting rods (35), and the space surrounded by the plurality of supporting connecting rods (35) is also provided with a data acquisition device (9), the data acquisition device (9) comprising a circumferentially movable track trolley (91) and a second industrial camera (94) mounted on the track trolley (91); The unfolding drive mechanism comprises a folding sleeve rod (34) connected between the supporting connecting rod (35) and the shifting rod (33), the folding sleeve rod (34) comprising a first folding rod (344) and a second folding rod (345), one end of the first folding rod (344) being hinged to the shifting rod (33), the other end of the first folding rod (344) being hinged to one end of the second folding rod (345), the other end of the second folding rod (345) being hinged to the supporting connecting rod (35), the second folding rod (345) being connected to a traction rope (32), and the unfolding drive mechanism further comprising a rope pulling mechanism for driving all the traction ropes (32) to be pulled up synchronously; The rope pulling mechanism comprises a plurality of bevel gear shafts arranged circumferentially and a third motor (39) for driving any bevel gear shaft to rotate, the ends of adjacent bevel gear shafts are connected via bevel gear transmission, and the traction rope (32) is wound around the bevel gear shaft.

2. A three-dimensional phenotyping robot for wheat in the field according to claim 1, characterized in that: The walking trolley comprises a frame (4) and supporting legs mounted at four corners of the frame (4), wherein the lower ends of the supporting legs are connected to a walking mechanism (1) via a steering device (2).

3. A three-dimensional phenotyping robot for wheat in the field according to claim 2, characterized in that: The support leg is connected to the vehicle frame (4) via a vehicle body adjustment device (5); the vehicle body adjustment device (5) comprises a width adjustment housing (51) fixedly connected to the vehicle frame (4) and an extrusion plate (54) laterally slidably connected to the width adjustment housing (51); the support leg is fixedly connected to the extrusion plate (54); one end of the width adjustment housing (51) is threadedly connected to an adjustment rod (52) pressed against one side of the extrusion plate (54); and the other end of the width adjustment housing (51) is provided with a compression spring (55) pressed against the other side of the extrusion plate (54).

4. The three-dimensional phenotyping robot for wheat in the field according to claim 1, characterized in that: The double guide rail mechanism (7) comprises an X-transverse guide rail (71) and a Y-longitudinal guide rail (73) vertically slidably connected to a slider of the X-transverse guide rail (71), and the three-dimensional phenotype analysis device (3) is fixedly connected to the lower end of the Y-longitudinal guide rail (73).

5. The three-dimensional phenotyping robot for wheat in the field according to claim 1, characterized in that: A downward-facing hydraulic rod (374) is fixedly connected inside the fixed housing (31), a slide table (37) is fixedly connected to the lower end of the hydraulic rod (374), and the first industrial camera (372) is mounted on the slide table (37).

6. The three-dimensional phenotyping robot for wheat in the field according to claim 1, characterized in that: An annular electrified soft rail (36) is installed in the space enclosed by the plurality of supporting connecting rods (35). The inner side of the supporting connecting rods (35) is connected to the electrified soft rail (36) via a track fixing frame (361). The track trolley (91) moves on the electrified soft rail (36).

7. The three-dimensional phenotyping robot for wheat in the field according to claim 1, characterized in that: An illumination lamp (348) is installed inside the supporting connecting rod (35).

8. A method for three-dimensional phenotyping of wheat in the field, using the three-dimensional phenotyping robot according to claim 1, characterized in that: The data acquisition device stops rotating after each 25-degree rotation, and the second industrial camera (94) takes photos, and the rotation continues. After one rotation, the acquired photos and the photos acquired by the first industrial camera (372) are transmitted to the edge computing device, and the point cloud is reconstructed using the improved colmap algorithm and the improved mvs algorithm, and then processed using the limit-based value-based straight-through filtering algorithm and the k-neighborhood-based bilateral filtering algorithm to obtain a complete and smooth three-dimensional point cloud; The improved colmap algorithm reconstructs the sparse point cloud of crops. The sparse point cloud reconstruction content is as follows: first extract the photo features, search for corresponding points, and extract the camera internal parameters from EXIF; then match the extracted features, and generate a scene graph and a matching matrix at the end of the process; then perform incremental modeling, gradually increase the viewing angle, and iteratively optimize the reprojection error to obtain the sparse point cloud of the scene and the camera posture of each viewing angle; finally remove the image distortion, and use optical consistency and geometric consistency to jointly match the bound structure value; The improved MVS algorithm reconstructs the dense point cloud of crops. The dense point cloud reconstruction content is as follows: first, the scene depth is estimated to obtain the depth value and normal vector value of the estimated perspective, and the depth optimization is performed using consistency; then the depth map is fused for dense reconstruction; then a coarse grid is established and the grid is refined; finally, the grid texture is mapped to obtain a relatively smooth dense point cloud; The through-filter algorithm based on the limit value removes the background point cloud: by setting thresholds in the three directions of X, Y, and Z respectively to enclose the point cloud cube, the target point cloud is enclosed in the bounding box, and the point cloud outside the threshold is deleted; The k-neighborhood-based bilateral filtering algorithm removes noise and performs smoothing. First, a kd tree is constructed for the point cloud to describe the location and existence attributes of the points, and then a k-neighborhood analysis is performed. Finally, bilateral filtering is used to remove noise points and the point cloud is smoothed to obtain a complete and smooth three-dimensional crop point cloud. Finally, the wheat ears, stems, leaves and non-plant point clouds are automatically segmented and the phenotypic parameters of the wheat plants are extracted.

Images