A standard fruit and vegetable garden multi-source sensor phenotype collection equipment and method

Abstract

This invention relates to the field of agricultural robots and multi-source sensor acquisition technology, and discloses a standard multi-source sensor phenotypic acquisition equipment and method for fruit and vegetable gardens. The equipment uses a quadruped robot body as a carrier, with a gimbal mounted on top of its back anti-vibration support frame. A lidar and an industrial camera are mounted on either side of the gimbal, and a radar, ultrasonic sensor, and motion camera are mounted on the head. The anti-vibration support frame has embedded silicone shock-absorbing pads, the gimbal contains a three-axis servo motor connected to an IMU, and an edge computing module is electrically connected to each sensor, forming a three-layer protection. The lidar, radar, and ultrasonic sensors extract terrain signals at near, medium, and far distances respectively, and fuse them with a weight of 0.5:0.3:0.2 to generate a terrain prediction signal. This signal is then fed forward with the IMU attitude disturbance vector with a weight of 0.5:0.5 to generate a gimbal pre-compensation control quantity. After the gimbal stabilizes, the sensors are triggered to acquire data synchronously. Obstacle signals at near, medium, and far distances are fused with the same weight. Self-locking is triggered only when the distance is <0.5m and the IMU tilt angle is >45°.

Description

Technical Field

[0001] This invention belongs to the field of agricultural robot and multi-source sensor acquisition technology, specifically a standard multi-source sensor phenotypic acquisition equipment and method for fruit and vegetable gardens. Background Technology

[0002] Existing field phenotypic data acquisition systems based on quadruped robots typically involve mounting a gimbal on the back of the quadruped robot via a vibration-damping support frame. The gimbal is equipped with a LiDAR and an industrial camera, and data is collected along the inspection path after factory calibration.

[0003] This system suffers from a unique drawback when conducting simultaneous multi-source sensor data acquisition in undulating terrain of fruit and vegetable gardens: the four stages—ground perception, gimbal stabilization, plant acquisition, and obstacle avoidance—operate independently. Ground perception is used only for navigation, the gimbal only responds to existing shaking, and plant acquisition and obstacle avoidance are performed by separate sensors with no communication between them. The physical mechanism of this drawback lies in the fact that when the lidar acquires point clouds of plant shapes, the same frame of point cloud output naturally includes both ground and obstacle point clouds. However, the existing architecture uses the point cloud data only for 3D reconstruction of the plant, filtering and discarding the ground point cloud as "useless data," and obstacle detection is performed by a separate sensor—different components of the same data source are fragmented and assigned to unconnected functional modules. Due to the lack of prior perception of the terrain ahead, the gimbal can only perform feedback compensation after pitch and sway have occurred, and cannot proactively adjust before the attitude deteriorates. Conventional methods are insufficient to resolve this contradiction: increasing the thickness of passive damping will raise the center of gravity and reduce stability; increasing the IMU feedback control bandwidth can only reduce response delay, but cannot achieve terrain prediction; adding a dedicated terrain sensor will increase system complexity and cost. The above defects only occur when there is a need for high-precision synchronous data acquisition from multiple sources on unstructured furrows in orchards and vegetables—the problem is not significant on flat roads or in scenarios with a single sensor.

[0004] To address these issues, this invention proposes a standard multi-source sensor phenotypic acquisition device and method for fruit and vegetable gardens. Summary of the Invention

[0005] In view of the above situation and to overcome the defects of the prior art, the present invention provides a standard multi-source sensor phenotypic acquisition equipment and method for fruit and vegetable gardens, which effectively solves the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a standard multi-source sensor phenotypic acquisition equipment for fruit and vegetable gardens, comprising a quadruped robot body, wherein the top of the quadruped robot body is provided with a support and protection mechanism, a rotating mechanism, a gimbal and an acquisition mechanism from bottom to top; The support and protection mechanism includes a vibration-damping support frame, inside which a high-elastic composite silicone shock-absorbing pad is embedded, forming the first layer of protection; The rotating mechanism includes a rotary motor, a rotary table is fixed to the outside of the output shaft of the rotary motor, and the top of the output shaft of the rotary table is fixedly connected to the gimbal. The gimbal is equipped with a three-axis servo motor and connected to the built-in IMU inertial measurement unit. The top of the gimbal is equipped with a dual-head pitch motor, forming a second layer of protection. The data acquisition mechanism includes a lidar unit, with an industrial camera mounted on the side of the lidar unit; It also includes an edge computing module, which is electrically connected to the LiDAR, industrial camera, and gimbal to form a third layer of protection. The system receives environmental point cloud signals from the lidar and extracts ground point cloud data. After generating the gimbal pre-compensation control quantity, it outputs the control quantity to the pitch dual-head motor to perform reverse pitch and the rotation motor to perform horizontal rotation. After the gimbal stabilizes, it triggers the LiDAR and industrial camera to simultaneously acquire 3D point cloud and RGB images, and continuously receives ground point cloud feedback signals through the LiDAR to update the terrain undulation index. Obstacle distance is extracted by receiving the same frame of laser point cloud signal from LiDAR. ,when When the value is less than the safety threshold and the IMU tilt angle signal exceeds the threshold, a self-locking signal is generated, which drives the gimbal to retract the LiDAR and industrial camera into the safety compartment and shuts off the power.

[0007] Preferably, the quadruped robot body has four robot thighs at its end, and each robot thigh has a lower leg hinged to its bottom; A battery is fixed to the side wall of the quadruped robot, a radar is fixed to the top of the quadruped robot, a moving camera is fixed to the head of the quadruped robot, and an ultrasonic sensor is fixed to the bottom of the head of the quadruped robot.

[0008] Preferably, the bottom of the vibration-damping support frame is fixed with a connecting plate, and bolts pass through several connecting holes on the connecting plate to be detachably connected to the quadruped robot body. The top of the vibration-damping support frame is detachably connected with a top frame. The rotary motor is fixedly connected to the top frame. A pitch hinge block is fixedly installed on the top of the gimbal. A pitch hinge plate is hinged to the outside of the pitch hinge block through a rotating shaft. The rotating shaft is fixedly connected to the pitch hinge plate and to the output shaft of the dual-head pitch motor.

[0009] Preferably, a lidar bracket is detachably connected to the top of the pitch hinge plate, the lidar bracket is fixedly connected to the lidar, an industrial camera is fixedly connected to the side wall of the lidar bracket, the industrial camera is fixedly connected to its internal industrial camera bracket, and a lens is detachably connected to the end of the industrial camera bracket.

[0010] This invention also provides a standard multi-source sensor phenotypic acquisition method for fruit and vegetable gardens. Based on the standard multi-source sensor phenotypic acquisition equipment for fruit and vegetable gardens described above, the acquisition method is executed by an edge computing module and includes the following steps: Step 1: The edge computing module extracts long-range ground point cloud signals from the LiDAR and extracts long-range terrain undulation indicators; it receives mid-range environmental echo signals from the radar and extracts mid-range terrain trend signals; and it receives near-range ground echo signals from the ultrasonic sensor and extracts near-range terrain elevation signals. These three signals are weighted and fused to generate a fused terrain prediction signal. The IMU receives three-axis angular velocity and acceleration signals and performs fusion calculations to generate a fuselage attitude disturbance vector. The fused terrain prediction signal and the fuselage attitude disturbance vector are then fed forward to generate a gimbal pre-compensation control quantity, which is output to the pitch dual-head motor to perform reverse pitch and to the rotation motor to perform horizontal rotation. Step 2: After the gimbal stabilizes, the synchronous triggering unit triggers the LiDAR, industrial camera, radar, mobile camera and ultrasonic sensor to synchronously collect multi-source data, and transmits the ground point cloud and ultrasonic distance signal back to update the fused terrain prediction signal. Step 3: Receive signals from distant obstacles using lidar and extract distance data for distant obstacles. It receives signals of mid-range obstacles via radar and extracts distance data of mid-range obstacles. It receives signals from nearby obstacles using ultrasonic sensors and extracts distance data from those obstacles. Through the edge computing module , , Weighted fusion generation ; Step 4: Utilize the edge computing module to... Combined with the IMU tilt signal, when When the tilt angle is less than the safety threshold and exceeds the threshold, a self-locking signal is generated, driving the pitch motor and rotary motor to retract the lidar and industrial camera into the safety compartment, and outputting a power-off signal to the power supply terminals of all sensors.

[0011] Preferably, a lidar is used to receive ground point cloud signals from a distance and calculate the height variance of the ground point cloud within a range of 0.5-2.0m in front. With slope angle A long-range terrain undulation index is generated. A medium-range terrain trend signal is generated by receiving medium-range environmental echo signals through radar 101 and performing coarse segmentation of point cloud. A short-range terrain elevation signal is generated by receiving short-range ground echo signals through ultrasonic sensors and performing distance calculation. The three are fused according to a weight of 0.5:0.3:0.2 to generate a fused terrain prediction signal. The terrain prediction signal and the fuselage attitude disturbance vector are combined into a gimbal pre-compensation control quantity with a weight of 0.5:0.5.

[0012] Preferably, , , Generate by weighted fusion with a ratio of 0.5:0.3:0.2. ; The synchronous triggering unit triggers the synchronous acquisition of LiDAR, industrial camera, radar, mobile camera and ultrasonic sensor at a frequency of 30Hz, and the acquired data is timestamped and stored through the ROS 2 framework.

[0013] Preferably, the current traveling speed v of the quadruped robot, obtained by integrating the IMU foot odometry, is read by the ARM processor of the edge computing module. When v < 0.3 m / s, the weight of the ultrasonic sensor is increased to 0.4 and the weight of the lidar is decreased to 0.3. When v > 0.6 m / s, the weight of the lidar is increased to 0.6 and the weight of the ultrasonic sensor is decreased to 0.1, so as to dynamically adjust the fusion weight of the terrain prediction signal.

[0014] Preferably, after solving for the extrinsic parameters of the initial rigid body transformation matrix using the singular value decomposition method with the calibration board image sequence and laser point cloud frame data, the results are substituted into the nonlinear optimization objective function for minimum reprojection error. Iterative optimization, in which These are the pixel coordinates of the calibration board feature points in the industrial camera image. Let K be the coordinates of the feature point in the 3D space of the lidar coordinate system, K be the intrinsic parameter matrix of the industrial camera, R be the rotation matrix, and t be the translation vector. The Levenberg-Marquardt algorithm is used for iteration. The iteration stops when the reprojection error is less than 0.5 pixels, and the optimized rigid body transformation matrix extrinsic parameters are output.

[0015] Preferably, the edge computing module generates a self-locking signal only when the safety threshold is 0.5m and the tilt angle threshold is 45°, and both must be satisfied simultaneously.

[0016] Compared with the prior art, the beneficial effects of the present invention are: (1) This device extracts terrain signals at near, medium, and long ranges using lidar, radar, and ultrasonic sensors with a weight of 0.5:0.3:0.2, and feeds them forward with the IMU attitude disturbance vector with a weight of 0.5:0.5. This enables the gimbal to perform reverse compensation before the aircraft's attitude changes, reducing the standard deviation of the pitch angle in the furrow terrain from ±1.8° to ±0.6° and the image reprojection error from 2.1 pixels to 0.7 pixels; at the same time, it reduces the obstacle distance... , , By integrating components with equal weight and linking them with the tilt angle signal for self-locking, the obstacle avoidance false trigger rate was reduced from 22% to 3%, thus solving the defect described in the background technology that the independent operation of each component could not form a causal closed loop. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0018] In the attached diagram: Figure 1 This is a schematic diagram of the overall device; Figure 2 This is a schematic diagram of the bottom of the entire device; Figure 3 This is a schematic diagram of the overall back of the device; Figure 4 This is a schematic diagram of the top of the vibration damping support frame of this device; Figure 5 This is a schematic diagram of the top of the rotating platform of this device.

[0019] In the diagram: 1-Quadruped robot body; 2-Robot thigh; 3-Vibration damping support frame; 4-Rotating table; 5-LiDAR bracket; 6-Industrial camera; 7-Rotating motor; 101-Radar; 102-Ultrasonic sensor; 103-Moving camera; 104-Battery; 201-Lower leg; 301-Connecting plate; 302-Connecting hole; 303-Top frame; 401-Gimbal; 402-Pitch hinge block; 403-Pitch hinge plate; 404-Dual pitch motor; 501-LiDAR; 601-Industrial camera bracket; 602-Lens. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] Example 1, by Figures 1-5The present invention discloses a standard multi-source sensor phenotypic acquisition device for fruit and vegetable gardens, comprising a quadruped robot body 1. The top of the quadruped robot body 1, from bottom to top, is provided with a support and protection mechanism, a rotation mechanism, a gimbal 401, and an acquisition mechanism. The support and protection mechanism includes a vibration-damping support frame 3, with a high-elastic composite silicone shock-absorbing pad embedded inside the frame 3, forming the first layer of protection. The rotation mechanism includes a rotary motor 7, with a rotary table 4 fixed externally to the output shaft of the rotary motor 7. The top of the output shaft of the rotary table 4 is fixedly connected to the gimbal 401. The gimbal 401 is equipped with a three-axis servo motor connected to an IMU (Inertial Measurement Unit) built into the body. A pitch dual-head motor 404 is located on the top of the gimbal 401, forming the second layer of protection. The acquisition mechanism... The system includes a lidar 501 with an industrial camera 6 mounted on its side; it also includes an edge computing module, which is electrically connected to the lidar 501, the industrial camera 6, and the gimbal 401, forming a third layer of protection. The lidar 501 receives environmental point cloud signals and extracts ground point clouds, generating pre-compensation control values ​​for the gimbal, which are then output to the pitch dual-head motor 404 for reverse pitch and the rotation motor 7 for horizontal rotation. Once the gimbal 401 stabilizes, it triggers the lidar 501 and the industrial camera 6 to simultaneously acquire 3D point clouds and RGB images, and continuously receives ground point cloud feedback signals from the lidar 501 to update terrain undulation indicators. The lidar 501 also receives the same frame of laser point cloud signals to extract obstacle distances. ,when When the value is less than the safety threshold and the IMU tilt angle signal exceeds the threshold, a self-locking signal is generated, which drives the gimbal 401 to retract the lidar 501 and the industrial camera 6 into the safety compartment and shuts off the power. In this embodiment, the silicone damping pad inside the anti-vibration support frame 3 is a ring structure with an outer diameter of 25mm, an inner diameter of 10mm, a thickness of 10mm, and a Shore hardness of 50. The gimbal 401 houses a three-axis servo motor connected to an inertial measurement unit (IMU) built into the robot body. The IMU outputs three-axis angular velocity and acceleration signals at a sampling frequency of 200Hz. The laser radar 501 has a point frequency of 240,000 points / second, and the industrial camera 6 has a frame rate of 30fps. The traveling speed v of the quadruped robot body 1 is obtained by integrating the IMU foot odometry at a sampling frequency of 100Hz. The edge computing module uses an NVIDIA Jetson AGX Orin, which includes an ARM Cortex-A78AE processor and an Ampere architecture GPU. The software framework is built on ROS 2 Humble. The maximum single-axis output limit of the gimbal pre-compensation control quantity is 15°. The duration of the self-locking signal is 500ms, and the signal is output through the edge computing module's I / O port to the enable terminal of the three-axis servo motor driver and the relay control terminal of the power control module.

[0022] Furthermore, the quadruped robot body 1 has four robot thighs 2 at its end, and each robot thigh 2 has a lower leg 201 hinged to its bottom; a battery 104 is fixed to the side wall of the quadruped robot body 1, a radar 101 is fixed to the top of the quadruped robot body 1, a moving camera 103 is fixed to the head of the quadruped robot body 1, and an ultrasonic sensor 102 is fixed to the bottom of the head of the quadruped robot body 1; a connecting plate 301 is fixed to the bottom of the vibration-damping support frame 3, and bolts pass through several connecting holes 302 on the connecting plate 301 to detachably connect to the quadruped robot body 1; a top frame 303 is detachably connected to the top of the vibration-damping support frame 3; The rotary motor 7 is fixedly connected to the top frame 303. The top of the gimbal 401 is fixedly equipped with a pitch hinge block 402. The pitch hinge block 402 is hinged to the outside of the pitch hinge plate 403 through a rotating shaft. The rotating shaft is fixedly connected to the pitch hinge plate 403 and to the output shaft of the pitch dual-head motor 404. The top of the pitch hinge plate 403 is detachably connected to a laser radar bracket 5. The laser radar bracket 5 is fixedly connected to the laser radar 501. An industrial camera 6 is fixedly connected to the side wall of the laser radar bracket 5. The industrial camera 6 is fixedly connected to the industrial camera bracket 601 inside it. A lens 602 is detachably connected to the end of the industrial camera bracket 601. In this embodiment, radar 101 is a millimeter-wave radar, model Continental ARS408-21, with a detection range of 0.5m-40m, used for mid-range terrain trend signal extraction. Ultrasonic sensor 102 is model HC-SR04, with a detection range of 2cm-450cm, outputting distance data at a frequency of 20Hz. Mobile camera 103 is model Intel RealSense D435, with a resolution of 1280×720, a field of view of 69°×42°, and outputting RGB images at 30fps. Industrial camera 6 is model Hikvision MV-CA050-10GC, with a resolution of 2448×2048, a frame rate of 30fps, and a lens 602 with a focal length of 12mm. LiDAR 501 is model Livox AVIA, with a detection range of 0.5m-100m, used for long-range ground point cloud and obstacle point cloud extraction. When using this device, the quadruped robot body 1 walks in the orchard and vegetable garden using its thighs 2 and calves 201. The impact generated by the feet hitting the ground is transmitted through the quadruped robot body 1 to the vibration-damping support frame 3. The silicone shock-absorbing pads inside the vibration-damping support frame 3 absorb high-frequency vibration energy through elastic deformation, blocking the transmission path of the shock wave to the gimbal 401, the lidar 501, and the industrial camera 6. At the same time, the three-axis servo motors inside the gimbal 401 perform reverse pitch and horizontal rotation compensation based on the body attitude disturbance signals detected in real time by the IMU, so that the lidar 501 and the industrial camera 6 maintain a relatively horizontal acquisition posture during walking. The lidar 501 holds a position during walking. The edge computing module continues to output the point cloud signal of the environment in front. After extracting the ground point cloud from the environmental point cloud to generate the terrain prediction signal, it combines the IMU attitude signal to generate the gimbal pre-compensation control quantity. It controls the pitch dual-head motor 404 to perform reverse pitch and the rotation motor 7 to perform horizontal rotation, so that the lidar 501 and the industrial camera 6 can adjust their acquisition attitude in advance to adapt to the changes in the terrain in front before the body attitude changes significantly. When the edge computing module detects that the quadruped robot body 1 falls, rolls over at an angle of more than 45° or has a sudden collision, it triggers the one-key self-locking logic. The gimbal 401 drives the lidar 501 and the industrial camera 6 to retract into the safety chamber in the vibration-damping support frame 3 and shuts off the power.

[0023] This embodiment presents a standard multi-source sensor phenotypic acquisition method for fruit and vegetable gardens. Based on the aforementioned standard multi-source sensor phenotypic acquisition equipment for fruit and vegetable gardens, the acquisition method is executed by an edge computing module and includes the following steps: Step 1: Extract long-range ground point cloud signals and long-range terrain undulation indicators using the edge computing module LiDAR 501; receive mid-range environmental echo signals and extract mid-range terrain trend signals using radar 101; receive near-range ground echo signals and extract near-range terrain elevation signals using ultrasonic sensor 102. The method for extracting long-range terrain undulation indicators is as follows: calculate the height variance of the ground point cloud within a range of 0.5-2.0m in front. With slope angle The extraction method for mid-range terrain trend signals is as follows: after voxel downsampling of the point cloud, the mean change rate of the ground point cloud elevation within a range of 2.0-5.0m in front is extracted; the extraction method for near-range terrain elevation signals is as follows: the mean value is obtained after median filtering of 5 consecutive frames of ultrasonic distance data. The three are then fused with weights of 0.5:0.3:0.2 to generate a fused terrain prediction signal; the fuselage attitude disturbance vector is generated by fusing and calculating the three-axis angular velocity and acceleration signals received by the IMU; the state vector of the Kalman filter is... ,in For pitch angle, For roll angle, Yaw angle For pitch angular velocity, For roll rate, Yaw angular velocity; process noise covariance matrix Set as Observation noise covariance matrix Set as The aircraft attitude disturbance vector is predicted and iterated at a frequency of 200Hz. The fused terrain prediction signal and the aircraft attitude disturbance vector are fed forward together with a weight of 0.5:0.5 to generate the gimbal pre-compensation control quantity, which is output to the pitch dual motor 404 to perform reverse pitch and to the rotation motor 7 to perform horizontal rotation. Step 2: After the gimbal 401 stabilizes, a synchronization pulse signal is emitted at a frequency of 30Hz through the synchronization trigger unit. This signal is then output via GPIO ports to the trigger inputs of the LiDAR 501, industrial camera 6, radar 101, mobile camera 103, and ultrasonic sensor 102. This triggers the LiDAR 501, industrial camera 6, radar 101, mobile camera 103, and ultrasonic sensor 102 to synchronously acquire multi-source data. The ground point cloud and ultrasonic distance signals are then transmitted back to the weighted fusion module from Step 1 to update the fused terrain prediction signal. The acquired data is timestamped and stored in rosbag format using the ROS 2 framework. The timestamps use the steady_clock time source, with a synchronization accuracy of 1ms. Step 3: Receive signals from distant obstacles using the LiDAR 501 and extract distance data for distant obstacles. (Extraction range > 5m) Receive mid-range obstacle signals and extract mid-range obstacle distance data via radar 101. (Extraction range is 1-5m), the ultrasonic sensor 102 receives the signal of the nearby obstacle and extracts the distance data of the nearby obstacle. (Extraction range is <1m) , , Outliers were removed by median filtering with a window size of 5 before fusion. The edge computing module then... , , Generate by linear weighted fusion with a weighting of 0.5:0.3:0.2. The fusion formula is ; Step 4: Utilize the edge computing module to... Jointly determine with the IMU tilt angle signal input edge calculation module, when Less than 0.5m and ( >45° or When both conditions (>45°) are met simultaneously, a self-locking signal is generated, driving the pitch dual-head motor 404 and the rotary motor 7 to retract the lidar 501 and industrial camera 6 into the safety compartment, and outputting a power-off signal to the power supply terminals of all sensors. Single-condition triggering (only) <0.5m but the inclination angle does not exceed the limit, or only the inclination angle exceeds the limit but When the depth is ≥0.5m, the self-locking action is not performed; instead, a warning log is sent to the cloud via the wireless communication module. By using the calibration board image sequence and laser point cloud frame data, the extrinsic parameters of the initial rigid body transformation matrix are solved using the singular value decomposition method, and then substituted into the nonlinear optimization objective function for minimum reprojection error. Iterative optimization is performed using the Levenberg-Marquardt algorithm. The iteration stops when the reprojection error is less than 0.5 pixels, and the optimized rigid body transformation matrix extrinsic parameters are output to achieve spatial alignment between the LiDAR 501 and the industrial camera 6. The ARM processor of the edge computing module reads the current walking speed v of the quadruped robot obtained by the IMU foot odometry integration. When v < 0.3 m / s, the weight of ultrasonic sensor 102 is increased to 0.4 and the weight of lidar 501 is decreased to 0.3. When v > 0.6 m / s, the weight of lidar 501 is increased to 0.6 and the weight of ultrasonic sensor 102 is decreased to 0.1, so as to dynamically adjust the fusion weight of the terrain prediction signal. When the quadruped robot moves at low speeds (<0.3 m / s), the contact time between its feet and the ground is longer, and the near-range ultrasonic echo is less affected by mechanical noise from the feet. Therefore, the ultrasonic weight is increased to enhance near-range elevation accuracy. When moving at high speeds (>0.6 m / s), the ground point cloud density decreases due to rapid movement, requiring long-range lidar to perceive terrain trends in advance. Therefore, the lidar weight is increased. This dynamic adjustment ensures that the fused terrain prediction signal maintains a stable phase lead at different speeds.

[0024] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A standard multi-source sensor phenotypic acquisition device for fruit and vegetable gardens, characterized in that: The quadruped robot body (1) includes a support and protection mechanism, a rotation mechanism, a gimbal (401) and a collection mechanism arranged sequentially from bottom to top on the top of the quadruped robot body (1); The support and protection mechanism includes a vibration-damping support frame (3), and the vibration-damping support frame (3) is embedded with a high-elastic composite silicone shock-absorbing pad to form the first layer of protection; The rotating mechanism includes a rotary motor (7), and a rotary table (4) is fixed to the outside of the output shaft of the rotary motor (7). The top of the output shaft of the rotary table (4) is fixedly connected to the gimbal (401). The gimbal (401) is equipped with a three-axis servo motor and connected to the built-in IMU inertial measurement unit. The top of the gimbal (401) is equipped with a pitch dual-head motor (404) to form a second layer of protection. The data acquisition mechanism includes a lidar (501), and an industrial camera (6) is provided on the side of the lidar (501). It also includes an edge computing module, which is electrically connected to the lidar (501), industrial camera (6), and gimbal (401) to form a third layer of protection; The laser radar (501) receives environmental point cloud signals and extracts ground point clouds. After generating the gimbal pre-compensation control quantity, it outputs it to the pitch dual-head motor (404) to perform reverse pitch and the rotation motor (7) to perform horizontal rotation. After the gimbal (401) stabilizes, it triggers the lidar (501) and industrial camera (6) to synchronously acquire three-dimensional point cloud and RGB image, and continuously receives ground point cloud feedback signals through lidar (501) to update terrain undulation indicators. Obstacle distance is extracted by receiving the same frame of laser point cloud signal from the lidar (501). ,when When the IMU tilt signal exceeds the safety threshold, a self-locking signal is generated, which drives the gimbal (401) to retract the lidar (501) and industrial camera (6) into the safety compartment and turn off the power.

2. The multi-source sensor phenotyping equipment for standard fruit and vegetable gardens according to claim 1, characterized in that: The quadruped robot body (1) has four robot thighs (2) at its end, and each robot thigh (2) has a lower leg (201) hinged to its bottom. A battery (104) is fixed to the side wall of the quadruped robot body (1), a radar (101) is fixed to the top of the quadruped robot body (1), a moving camera (103) is fixed to the head of the quadruped robot body (1), and an ultrasonic sensor (102) is fixed to the bottom of the head of the quadruped robot body (1).

3. The standard multi-source sensor phenotyping equipment for fruit and vegetable gardens according to claim 2, characterized in that: The bottom of the vibration-damping support frame (3) is fixed with a connecting plate (301), and bolts pass through several connecting holes (302) on the connecting plate (301) to be detachably connected to the quadruped robot body (1). The top of the vibration-damping support frame (3) is detachably connected with a top frame (303). The rotary motor (7) is fixedly connected to the top frame (303). The top of the gimbal (401) is fixed with a pitch hinge block (402). The pitch hinge block (402) is hinged to the outside of the pitch hinge plate (403) through a rotating shaft. The rotating shaft is fixedly connected to the pitch hinge plate (403). The rotating shaft is fixedly connected to the output shaft of the pitch dual-head motor (404).

4. The multi-source sensor phenotyping equipment for standard fruit and vegetable gardens according to claim 3, characterized in that: The top of the pitch hinge plate (403) is detachably connected to a laser radar bracket (5), the laser radar bracket (5) is fixedly connected to the laser radar (501), the side wall of the laser radar bracket (5) is fixedly connected to an industrial camera (6), the industrial camera (6) is fixedly connected to the industrial camera bracket (601) inside it, and the end of the industrial camera bracket (601) is detachably connected to a lens (602).

5. A method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden, based on the multi-source sensor phenotypic data acquisition equipment for a standard fruit and vegetable garden as described in any one of claims 1-4, characterized in that: The data acquisition method is executed by the edge computing module and includes the following steps: Step 1: Extract long-range ground point cloud signals and long-range terrain undulation indicators through the edge computing module lidar (501), receive mid-range environmental echo signals and extract mid-range terrain trend signals through radar (101), receive near-range ground echo signals and extract near-range terrain elevation signals through ultrasonic sensor (102), and weight and fuse the three to generate a fused terrain prediction signal; receive three-axis angular velocity and acceleration signals through IMU and generate a fuselage attitude disturbance vector through fusion calculation; combine the fused terrain prediction signal and the fuselage attitude disturbance vector with feedforward fusion to generate a gimbal pre-compensation control quantity, output to the pitch dual-head motor (404) to perform reverse pitch, and output to the rotation motor (7) to perform horizontal rotation; Step 2: After the gimbal (401) stabilizes, the synchronous triggering unit triggers the lidar (501), industrial camera (6), radar (101), mobile camera (103) and ultrasonic sensor (102) to synchronously collect multi-source data, and transmits the ground point cloud and ultrasonic distance signal back to update the fused terrain prediction signal; Step 3: Receive signals of distant obstacles using the lidar (501) and extract distance data of distant obstacles. The radar (101) receives signals of mid-range obstacles and extracts distance data of mid-range obstacles. The ultrasonic sensor (102) receives signals from nearby obstacles and extracts distance data of the nearby obstacles. Through the edge computing module , , Weighted fusion generation ; Step 4: Utilize the edge computing module to... Combined with the IMU tilt signal, when When the tilt angle of the IMU exceeds the safety threshold, a self-locking signal is generated, which drives the pitch dual-head motor (404) and the rotary motor (7) to retract the lidar (501) and the industrial camera (6) into the safety chamber, and outputs a power-off signal to the power supply terminals of all sensors.

6. The method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden according to claim 5, characterized in that: The system uses a lidar (501) to receive ground point cloud signals from a distance and calculates the height variance of the ground point cloud within a range of 0.5-2.0m in front. With slope angle A long-range terrain undulation index is generated. A medium-range terrain trend signal is generated by receiving a medium-range environmental echo signal through radar (101) and coarsely segmenting the point cloud. A short-range terrain elevation signal is generated by receiving a short-range ground echo signal through an ultrasonic sensor (102) and calculating the distance. The three are fused according to a weight of 0.5:0.3:0.2 to generate a fused terrain prediction signal. The terrain prediction signal and the fuselage attitude disturbance vector are combined into a gimbal pre-compensation control quantity with a weight of 0.5:0.

5.

7. The method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden according to claim 6, characterized in that: Will , , Generate by weighted fusion with a ratio of 0.5:0.3:0.

2. ; The LiDAR (501), industrial camera (6), radar (101), mobile camera (103) and ultrasonic sensor (102) are triggered synchronously at a frequency of 30Hz by the synchronous triggering unit, and the acquired data is timestamped and stored through the ROS 2 framework.

8. The method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden according to claim 7, characterized in that: The ARM processor of the edge computing module reads the current walking speed v of the quadruped robot obtained by the IMU foot odometry integration. When v < 0.3 m / s, the weight of the ultrasonic sensor (102) is increased to 0.4 and the weight of the lidar (501) is decreased to 0.

3. When v > 0.6 m / s, the weight of the lidar (501) is increased to 0.6 and the weight of the ultrasonic sensor (102) is decreased to 0.1, so as to dynamically adjust the fusion weight of the terrain prediction signal.

9. The method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden according to claim 8, characterized in that: By using the calibration board image sequence and laser point cloud frame data, the extrinsic parameters of the initial rigid body transformation matrix are solved using the singular value decomposition method, and then substituted into the nonlinear optimization objective function for minimum reprojection error. Iterative optimization, in which The pixel coordinates of the calibration board feature points in the image of the industrial camera (6) are given. Let K be the coordinates of the feature point in the three-dimensional space of the lidar (501) coordinate system, K be the intrinsic parameter matrix of the industrial camera (6), R be the rotation matrix, and t be the translation vector. The Levenberg-Marquardt algorithm is used for iteration. When the reprojection error is less than 0.5 pixels, the iteration stops and the optimized rigid body transformation matrix extrinsic parameters are output.

10. A method for acquiring phenotypic data using a multi-source sensor in a standard fruit and vegetable garden according to claim 9, characterized in that: The edge computing module generates a self-locking signal only when the safety threshold is 0.5m and the tilt angle threshold is 45°, and both conditions must be met simultaneously.

Images