A lychee contour-following vibration harvesting machine and its control method

Abstract

This invention discloses a lychee contour-following vibrating harvester and its control method. The harvester includes a transport mechanism, a navigation mechanism, a multimodal sensing mechanism, a harvesting device, a fruit collection mechanism, and a lifting and translating contour-following mechanism mounted on the transport mechanism. The lifting and translating contour-following mechanism includes a moving support, a translating mechanism, a lifting mechanism, and a contour-following mechanism. The contour-following mechanism is used to adjust the harvesting angle of the harvesting device. The lifting mechanism is used to drive the contour-following mechanism to perform vertical movement. The translating mechanism is used to drive the moving support to perform horizontal movement. The harvesting device includes a harvesting frame mounted on the contour-following mechanism, a gathering mechanism mounted on the harvesting frame, and a vibrating harvesting execution mechanism. This harvester greatly reduces the difficulty of operation for users, improves the harvesting accuracy of the harvester through precise posture adjustment, increases the working efficiency of the harvester, and thus improves the harvesting efficiency of lychees. It has a high degree of automation and intelligence.

Description

Technical Field

[0001] This invention relates to the field of fruit harvesting equipment technology, specifically to a lychee contour-following vibrating harvester and its control method. Background Technology

[0002] Lychee is one of my country's "Four Major Fruits of Southern China," and my country ranks first in the world in both planting area and output. At the same time, lychee is also an important economic crop in southern China. Lychee is also widely used to produce fruit by-products, such as canned lychee in syrup and preserved lychee. For lychee used in these products, the harvesting process does not require significant mechanical damage to the fruit.

[0003] Currently, lychee harvesting is mainly done manually, which is not only labor-intensive and inefficient, but also results in lychees having a concentrated ripening period and a short shelf life. If not harvested and sold in time, they are prone to sugar loss, fruit drop, and rotting. Furthermore, with the continuous rise in labor costs, there is a risk of increased production without increased income, which does not meet the interests of fruit farmers.

[0004] Automated harvesting of lychees has become an urgent need for the development of the lychee industry. Although mechanized harvesting technology can easily cause mechanical damage to the fruit, affecting the commercialization and sales price of fresh fruit, it has broad application prospects for fruit by-products used in post-processing and is of great significance for achieving cost reduction and efficiency improvement in the industry.

[0005] With the promotion of agricultural mechanization, more and more machines are being used in orchard harvesting. However, the level of intelligence in lychee harvesting machinery on the market is relatively low, and some even require precise manual operation, demanding certain operating skills and limiting the improvement of work efficiency to some extent. Moreover, the requirements for harvesting machinery are relatively high. When the computing power of the harvesting machinery is insufficient, the efficiency of fruit harvesting is low, and the harvesting accuracy cannot meet the requirements, even being less efficient than manual harvesting. Summary of the Invention

[0006] The purpose of this invention is to overcome the above-mentioned problems and provide a lychee contour-following vibration harvester. This harvester greatly reduces the difficulty of operation for users, improves the harvesting accuracy of the harvester through precise posture adjustment, improves the working efficiency of the harvester, and thus improves the harvesting efficiency of lychees. It has a high degree of automation and intelligence.

[0007] Another objective of this invention is to provide a control method for a lychee contour-following vibration harvesting machine.

[0008] The objective of this invention is achieved through the following technical solution:

[0009] A lychee contour-following vibrating harvesting machine includes a transport mechanism for movement, a navigation mechanism mounted on the transport mechanism for positioning and navigation, a multimodal sensing mechanism for identifying the lychee tree canopy, a harvesting device for harvesting lychee fruits, a fruit collecting mechanism for collecting lychee fruits, and a lifting and translating contour-following mechanism for driving the harvesting device to move; wherein,

[0010] The lifting and translating contouring mechanism includes a movable support, a translation mechanism, a lifting mechanism, and a contouring mechanism; wherein, the movable support is horizontally slidably disposed on the top of the transport mechanism; the contouring mechanism is vertically slidably disposed on the movable support and is used to adjust the harvesting angle of the harvesting device; the lifting mechanism is disposed on the movable support and is used to drive the contouring mechanism to perform vertical movement to drive the harvesting device to perform vertical movement; the translation mechanism is used to drive the movable support to perform horizontal movement to drive the harvesting device to perform horizontal movement.

[0011] The harvesting device includes a harvesting frame mounted on the contouring mechanism, a gathering mechanism mounted on the harvesting frame for gathering the canopy of the litchi tree, and a vibrating harvesting execution mechanism; wherein, the multimodal sensing mechanism is mounted on the movable support, and the fruit collecting mechanism is located below the vibrating harvesting execution mechanism.

[0012] The working principle of the above-mentioned lychee contour-following vibration harvesting machine is as follows:

[0013] The harvester moves to the designated position via a navigation mechanism. A lifting, translating, and contouring mechanism drives the harvesting device to ensure it reaches the appropriate height, distance, and orientation (angle) for harvesting. A multimodal sensing mechanism identifies the litchi tree canopy and obtains information about the litchi fruits to be harvested. Based on this information, the contouring mechanism adjusts the angle of the harvesting device to achieve the optimal harvesting posture. A gathering mechanism gathers the branches of the litchi tree canopy, and a vibration harvesting actuator taps the branches, causing them to vibrate and fall off, thus harvesting the litchis. Finally, the fallen litchis are collected by a fruit collection mechanism.

[0014] In a preferred embodiment of the present invention, the vibratory harvesting actuator includes a vibratory tapping mechanism mounted on the harvesting frame and a crank-connecting rod drive mechanism for driving the vibratory tapping mechanism. The vibratory tapping mechanism includes vibratory tapping assemblies mounted on both sides of the harvesting frame. Each vibratory tapping assembly includes a harvesting roller rotatably mounted on the harvesting frame and multiple tapping rods mounted on the harvesting roller. The crank-connecting rod drive mechanism is connected to the two harvesting rollers. In this structure, during harvesting, the crank-connecting rod drive mechanism drives the two harvesting rollers to rotate back and forth, thereby causing the tapping rods on the harvesting rollers to swing back and forth, tapping the branches. The branches vibrate, causing the lychee fruits on the branches to fall off, thus achieving harvesting. By mounting vibratory tapping assemblies on both sides of the harvesting frame (i.e., two sets of vibratory tapping assemblies), the tapping effect is improved, and the back-and-forth tapping of the branches makes it easier for the lychee fruits to fall off.

[0015] Preferably, the crank-connecting rod drive mechanism includes a vibratory beating drive motor, a crank, a first intermediate connecting rod, a first rocker arm, a second intermediate connecting rod, and a second rocker arm. The vibratory beating drive motor is located at the lower end of the harvesting frame. One end of the crank is connected to the main shaft of the vibratory beating drive motor. One end of the first intermediate connecting rod and one end of the second intermediate connecting rod are connected to the other end of the crank. One end of the first rocker arm is connected to the other end of the first intermediate connecting rod, and the other end of the first rocker arm is connected to one of the harvesting rollers. One end of the second rocker arm is connected to the other end of the second intermediate connecting rod, and the other end of the second rocker arm is connected to another harvesting roller. In this structure, during harvesting, the vibratory beating drive motor drives the rocker arm to rotate, causing the first intermediate connecting rod and the second intermediate connecting rod to move, which in turn causes the first and second rocker arms to swing back and forth, thereby rotating back and forth to achieve the beating action of the beating rod.

[0016] Preferably, the gathering mechanism includes two gathering arms disposed opposite to each other on the harvesting frame; each gathering arm includes a first gathering plate, a second gathering plate, a first gathering drive motor, and a second gathering drive motor; the first gathering plate is disposed between the harvesting frame and the second gathering plate, one end of the first gathering plate is rotatably connected to the harvesting frame, and the other end is rotatably connected to the end of the second gathering plate; the first gathering drive motor is used to drive the first gathering plate to rotate on the harvesting frame, and the second gathering drive motor is used to drive the second gathering plate to rotate on the first gathering plate. The aforementioned gathering arms, similar to arms, gather the branches of the lychee tree canopy by bringing them together. This facilitates the vibration harvesting mechanism's tapping action, improving harvesting efficiency. Specifically, the two gathering arms move synchronously inward or outward. When gathering, the first gathering drive motor drives the first gathering plate to rotate inward, and the second gathering plate also rotates inward. The second gathering drive motor then drives the second gathering plate to rotate inward, further gathering the branches of the lychee tree canopy. After this, both the first and second gathering drive motors stop driving. When opening, the first gathering drive motor drives the first gathering plate to rotate outward, and the second gathering plate also rotates outward. The second gathering drive motor then drives the second gathering plate to rotate outward, further opening the gathering arms and loosening the branches of the lychee tree canopy.

[0017] Preferably, the lychee contour-following vibrating harvester further includes a control cabinet mounted on the transport mechanism. The navigation mechanism includes a first lidar and a GPS. The first lidar is mounted on the control cabinet to provide high-precision environmental perception information. The GPS is located at both ends of the control cabinet to provide the position and speed information of the transport mechanism. By acquiring high-precision environmental perception information and the position and speed information of the transport mechanism, and fusing these data together, comprehensive and accurate environmental perception and positioning navigation information is provided for the autonomous driving system. The control cabinet can control the movement of the transport mechanism and also control the harvesting device to perform harvesting actions based on the environmental perception and positioning navigation information.

[0018] Preferably, the multimodal sensing mechanism includes gimbal support frames disposed on both sides of the movable support, a camera disposed at the end of one of the gimbal support frames, and a second lidar disposed at the end of the other gimbal support frame. In the above structure, after calibration by the second lidar and the camera, three-dimensional point cloud information of the litchi tree canopy and the position information of the litchi fruits are obtained. After the harvesting device moves to the position of the litchi fruits to be harvested, the control cabinet processes the three-dimensional point cloud information to obtain the three-dimensional information of the canopy near the litchi fruits to be harvested, calculates and judges the optimal harvesting angle, and converts the harvesting angle into an electrical signal to control the contouring mechanism to adjust the contouring angle of the harvesting device.

[0019] Preferably, the contouring mechanism includes a contouring support plate vertically slidably mounted on the movable support and an electric push rod disposed between the contouring support plate and the harvesting frame; the contouring support plate is rotatably connected to the harvesting frame, one end of the electric push rod is hinged to the contouring support plate, the other end of the electric push rod is hinged to the harvesting frame, and the contouring support plate is connected to the lifting mechanism. In the above structure, the contouring mechanism realizes the tilt angle change of the harvesting device (0°~120°) through the electric push rod. By extending and retracting the electric push rod, the harvesting frame is pushed to swing, causing the angle between the contouring support plate and the harvesting frame to change, thereby changing the angle of the entire harvesting device; in the initial state, the harvesting frame is at 90° to the ground, that is, parallel to the contouring support plate; when the electric push rod extends, the harvesting frame changes from 90° to 0°, and when the electric push rod retracts, the harvesting frame changes from 90° to 120°.

[0020] Preferably, the lychee contour-following vibrating harvester further includes a blower mechanism mounted on the movable support; the blower mechanism includes a fan mounting frame installed on the movable support and multiple blowers arranged side by side on the fan mounting frame. During the harvesting process, not only lychee fruits fall off when the fruit is tapped, but also lychee leaves. The blower mechanism can blow away lychee leaves and small branches during the harvesting process, preventing lychee leaves and small branches from falling into the fruit collection mechanism.

[0021] Preferably, the translation mechanism includes a slide rail, a translation drive motor, and a translation screw and nut assembly disposed on the top of the transport mechanism; the moving support is slidably disposed on the slide rail, the screw and nut of the translation screw and nut assembly are connected to the moving support, and the translation drive motor is connected to the screw of the translation screw and nut assembly; the lifting mechanism includes a lifting drive motor and a lifting screw and nut assembly; the lifting drive motor is mounted on the upper end of the moving support, the screw and nut of the lifting screw and nut assembly are connected to the contour support plate, and the screw of the lifting screw and nut assembly is connected to the lifting drive motor. During harvesting, the translation drive motor drives the translation screw and nut assembly to move, causing the moving support to move horizontally, thereby causing the contour mechanism and the harvesting device to move horizontally together; the lifting drive motor drives the lifting screw and nut assembly to move, causing the contour mechanism and the harvesting device to move vertically together. Through the cooperation of the lifting mechanism, the translation mechanism, and the contour mechanism, the harvesting device is ensured to perform harvesting operations at an appropriate height and distance.

[0022] Preferably, the fruit collecting mechanism includes a lychee harvesting frame mounted on the slide rail and a guide enclosure frame positioned between the harvesting device and the lychee harvesting frame, the guide enclosure frame being connected to the movable support. In this structure, the guide enclosure frame guides the lychee fruit to fall into the lychee harvesting frame. The lychee harvesting frame can move horizontally synchronously with the movable support, ensuring that the horizontal positions of the lychee harvesting frame and the guide enclosure frame remain unchanged, facilitating the lychee fruit to fall into the lychee harvesting frame under the guidance of the guide enclosure frame. The guide enclosure frame can be connected to the movable support via a screw drive element, which can drive the guide enclosure frame to move vertically. Additionally, the lifting screw nut pair can have two screw nuts, and the guide enclosure frame can also be connected to one of the screw nuts in the lifting screw nut pair, achieving synchronous lifting and lowering movement of the guide enclosure frame and the harvesting device.

[0023] A control method for a lychee contour-following vibration harvesting machine includes the following steps:

[0024] (1) Positioning and navigation of navigation institutions:

[0025] The first lidar acquires high-precision environmental perception information, and the GPS acquires the position and speed information of the vehicle. The acquired environmental perception information, position and speed information are used to locate the vehicle, build a map and plan a path. The vehicle then reaches the designated location according to the planned path.

[0026] (2) The lifting mechanism and the translation mechanism perform lifting and translation:

[0027] The lifting mechanism adjusts the height of the harvesting device, and the translation mechanism adjusts the distance between the harvesting device and the lychee fruit, ensuring that the harvesting device can be used for harvesting operations at a suitable height and distance.

[0028] (3) Dataset creation:

[0029] Multiple images of the litchi tree canopy were acquired using a camera from a multimodal sensing mechanism. Then, the acquired images of the litchi tree canopy were labeled using a labeling tool to select the litchi tree canopy and the litchi fruits to be picked in the canopy. This resulted in a litchi tree canopy and litchi fruit dataset.

[0030] (4) Constructing deep learning models:

[0031] The YOLO v9 network was used to train the litchi fruit dataset to obtain a network model that can identify the litchi tree canopy and litchi fruit.

[0032] (5) Joint calibration of the second lidar and camera to obtain the conversion relationship between the second lidar and camera data, and find the corresponding pixel points in the lidar point cloud data and the litchi tree canopy image at the same time.

[0033] (6) Time registration of the second lidar and camera:

[0034] The image data with the smallest time interval with each frame of the second lidar data is found by using the time nearest neighbor matching method and then processed to achieve time registration between the second lidar and the camera.

[0035] (7) Three-dimensional reconstruction:

[0036] The color image of the litchi fruit acquired by the camera is input into the network model that has been trained in step (4) to obtain the location coordinate information of the litchi fruit picking, and the three-dimensional point cloud of the litchi fruit is obtained according to the calibration and registration results of steps (5) and (6).

[0037] (8) Semantic segmentation for 3D data:

[0038] The direct processing method is adopted, and the Point-Net network is used to transform the three-dimensional point cloud obtained in step (7) into a regular 3D voxel mesh or image set. The Point-Net network performs feature extraction through learning, extracts local and global features from the original three-dimensional point cloud data, and captures the semantic relationship and spatial structure between different categories.

[0039] Then, the Point-Net network outputs the semantic category prediction for each point or voxel, and assigns it to the corresponding semantic labels of the fruit tree canopy and lychee fruit to achieve semantic segmentation of the 3D point cloud data.

[0040] (9) Motion trajectory planning:

[0041] The semantically segmented 3D point cloud of the lychee fruit is transmitted to the control cabinet. The control cabinet analyzes the spatial 3D information contained in the 3D point cloud and uses an obstacle avoidance algorithm to plan the motion trajectory of the lychee contour vibrating harvester so that it can get closer to the harvesting target.

[0042] (10) Harvesting posture adjustment:

[0043] The 3D point cloud of the litchi fruit obtained by semantic segmentation is used to determine the direction vector n of the litchi fruit cluster axis; the direction vector n is transmitted to the contouring mechanism, and contouring is performed from the direction of the direction vector n.

[0044] (11) Harvesting and Collection:

[0045] The harvesting target is surrounded by a closing mechanism, and the vibrating harvesting mechanism harvests the target by vibrating in the direction of direction vector n. The harvested lychee fruits fall into the fruit collection mechanism for collection.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. The litchi contour-following vibrating harvester of this invention moves to a designated position via a navigation mechanism. A lifting mechanism, a translation mechanism, and a contour-following mechanism drive the harvesting device to ensure it reaches the appropriate height, distance, and posture (angle) for harvesting. During harvesting, a gathering mechanism gathers the litchi tree canopy, specifically converging the branches. Then, a vibrating harvesting mechanism beats the branches, causing them to vibrate and the litchi fruits to fall off, thus achieving harvesting. Finally, the litchi fruits fall to a fruit collection mechanism for collection. This significantly reduces the user's operational difficulty, improves the harvester's efficiency, and consequently increases the litchi harvesting efficiency, demonstrating a high degree of automation and intelligence.

[0048] 2. The litchi contour-following vibration harvester of the present invention identifies the litchi tree canopy through a multimodal sensing mechanism, obtains information about the litchi fruits to be harvested in the litchi tree canopy, and adjusts the angle of the harvesting device according to the litchi fruit information so that the harvesting device is in the most suitable harvesting posture, thereby improving the harvesting accuracy of the harvester.

[0049] 3. The control method of the litchi contour-following vibrating harvester in this invention uses machine vision to identify and detect litchi fruits. It obtains three-dimensional point cloud information of the litchi tree canopy and litchi fruits through the joint calibration of the second lidar and camera, constructs the outline of the litchi tree canopy and obtains the spatial three-dimensional coordinate information of the litchi fruits, adjusts the position of the harvesting device according to the three-dimensional coordinates of the litchi fruits, calculates through the control cabinet, makes judgments and decisions on the next harvesting operation and executes it, drives the contour-following mechanism, adjusts the angle between the harvesting device and the litchi tree canopy, and realizes automatic contour-following harvesting of litchi fruits. Attached Figure Description

[0050] Figure 1 This is a front view of one specific embodiment of a lychee contour-following vibrating harvester of the present invention.

[0051] Figure 2 This is a right view of the litchi contour-following vibrating harvester of the present invention.

[0052] Figure 3 This is a perspective view of the litchi contour-following vibrating harvester of the present invention.

[0053] Figure 4 This is a schematic diagram of the installation structure of the harvesting device and the contouring mechanism in this invention.

[0054] Figure 5 This is a perspective view of the vibration harvesting actuator in this invention.

[0055] Figure 6 This is a perspective view of the closing arm in this invention.

[0056] Figure 7 This is a three-dimensional structural diagram of the control cabinet and navigation mechanism in this invention.

[0057] Figure 8 This is a schematic diagram of the installation structure of the harvesting frame and the contouring mechanism in this invention.

[0058] Figure 9 This is a flowchart of the camera fusion recognition algorithm of the second lidar in this invention.

[0059] Figure 10 This is a flowchart of the harvesting process of the litchi contour vibrating harvester of the present invention.

[0060] Figure 11 This is a diagram illustrating the transformation process between lidar coordinates and camera coordinates in this invention. Detailed Implementation

[0061] To enable those skilled in the art to fully understand the technical solutions of the present invention, the present invention will be further described below in conjunction with embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0062] Example 1

[0063] See Figures 1-3 This embodiment discloses a lychee contour-following vibrating harvesting machine, including a transport mechanism 1 for walking, a navigation mechanism disposed on the transport mechanism 1 for positioning and navigation, a multimodal sensing mechanism for identifying the canopy of lychee trees, a harvesting device for harvesting lychee fruits, a fruit collecting mechanism for collecting lychee fruits, and a lifting and translating contour-following mechanism for driving the harvesting device to move.

[0064] See Figures 1-3 The lifting and translating contouring mechanism includes a movable support 2, a translation mechanism 3, a lifting mechanism 4, and a contouring mechanism 5; wherein, the movable support 2 is horizontally slidably disposed on the top of the transport mechanism 1; the contouring mechanism 5 is vertically slidably disposed on the movable support 2 and is used to adjust the harvesting angle of the harvesting device; the lifting mechanism 4 is disposed on the movable support 2 and is used to drive the contouring mechanism 5 to perform vertical movement so as to drive the harvesting device to perform vertical movement; the translation mechanism 3 is used to drive the movable support 2 to perform horizontal movement so as to drive the harvesting device to perform horizontal movement.

[0065] See Figures 1-4 The harvesting device includes a harvesting frame 6 mounted on the contouring mechanism 5, a gathering mechanism 7 mounted on the harvesting frame 6 for gathering the canopy of the litchi tree, and a vibrating harvesting execution mechanism 8; wherein, the multimodal sensing mechanism is mounted on the movable support 2, and the fruit collection mechanism is mounted below the vibrating harvesting execution mechanism 8.

[0066] See Figures 1-5 The vibratory harvesting mechanism 8 includes a vibratory beating mechanism mounted on the harvesting frame 6 and a crank 8-4 connecting rod drive mechanism for driving the vibratory beating mechanism. The vibratory beating mechanism includes vibratory beating assemblies mounted on both sides of the harvesting frame 6. Each vibratory beating assembly includes a harvesting roller 8-1 rotatably mounted on the harvesting frame 6 and multiple beating rods 8-2 mounted on the harvesting roller 8-1. The crank 8-4 connecting rod drive mechanism is connected to the two harvesting rollers 8-1. In this structure, during harvesting, the crank 8-4 connecting rod drive mechanism drives the two harvesting rollers 8-1 to rotate back and forth, thereby causing the beating rods 8-2 on the harvesting rollers 8-1 to swing back and forth, beating the branches. The branches vibrate, causing the lychee fruits on the branches to fall off, thus achieving harvesting. By mounting vibratory beating assemblies on both sides of the harvesting frame 6 (i.e., two sets of vibratory beating assemblies), the purpose is to improve the beating effect, making it easier for the lychee fruits to fall off the branches.

[0067] See Figures 1-5 The vibrating beater assembly is a comb-shaped vibrating beater assembly, with multiple beaters 8-2 evenly arranged along the axial direction of the harvesting roller 8-1; the beaters 8-2 are rubber rods. The rubber rods can prevent damage to the litchi fruit during beating.

[0068] See Figures 1-5 The crank 8-4 connecting rod drive mechanism includes a vibration and tapping drive motor 8-3, a crank 8-4, a first intermediate connecting rod 8-5, a first rocker arm 8-6, a second intermediate connecting rod 8-7, and a second rocker arm 8-8. The vibration and tapping drive motor 8-3 is located at the lower end of the harvesting frame 6. One end of the crank 8-4 is connected to the main shaft of the vibration and tapping drive motor 8-3. One end of the first intermediate connecting rod 8-5 and one end of the second intermediate connecting rod 8-7 are connected to the other end of the crank 8-4. One end of the first rocker arm 8-6 is connected to the other end of the first intermediate connecting rod 8-5, and the other end of the first rocker arm 8-6 is connected to one of the harvesting rollers 8-1. One end of the second rocker arm 8-8 is connected to the other end of the second intermediate connecting rod 8-7, and the other end of the second rocker arm 8-8 is connected to the other harvesting roller 8-1. In the above structure, during harvesting, the vibration and beating drive motor 8-3 drives the rocker arm to rotate, which in turn moves the first intermediate connecting rod 8-5 and the second intermediate connecting rod 8-7, thereby causing the first rocker arm 8-6 and the second rocker arm 8-8 to swing back and forth, and thus rotate back and forth, realizing the beating of the beating rod 8-2.

[0069] See Figure 5The first intermediate connecting rod 8-5 and the second intermediate connecting rod 8-7 have different lengths, as do the first rocker arm 8-6 and the second rocker arm 8-8. This is to prevent the two vibrating beaters from moving synchronously, thus improving the beating effect and increasing the recovery rate. The two vibrating beaters are comb-shaped vibrating beaters, with their beating rods 8-2 staggered to prevent interference and collisions during the beating process.

[0070] See Figures 1-4 and Figure 6 The gathering mechanism 7 includes two gathering arms disposed opposite to each other on the harvesting frame 6; each gathering arm includes a first gathering plate 7-1, a second gathering plate 7-2, a first gathering drive motor 7-3, and a second gathering drive motor 7-4; the first gathering plate 7-1 is disposed between the harvesting frame 6 and the second gathering plate 7-2, one end of the first gathering plate 7-1 is rotatably connected to the harvesting frame 6, and the other end is rotatably connected to the end of the second gathering plate 7-2; the first gathering drive motor 7-3 is used to drive the first gathering plate 7-1 to rotate on the harvesting frame 6, and the second gathering drive motor 7-4 is used to drive the second gathering plate 7-2 to rotate on the first gathering plate 7-1. The aforementioned gathering arms resemble arms. By using two gathering arms to gather the branches of the lychee tree canopy, they can be easily tapped by the vibrating harvesting actuator 8, thus improving harvesting efficiency. Specifically, the two gathering arms move inward or outward simultaneously. When gathering, the first gathering drive motor 7-3 drives the first gathering plate 7-1 to rotate inward, and the second gathering plate 7-2 also rotates inward. The second gathering drive motor 7-4 drives the second gathering plate 7-2 to rotate inward, causing the gathering arms to further gather inward. After gathering the branches of the lychee tree canopy, the first gathering drive motor 7-3 and the second gathering drive motor 7-4 stop driving. When opening, the first gathering drive motor 7-3 drives the first gathering plate 7-1 to rotate outward, and the second gathering plate 7-2 also rotates outward. The second gathering drive motor 7-4 drives the second gathering plate 7-2 to rotate outward, causing the gathering arms to further open outward, loosening the branches of the lychee tree canopy.

[0071] See Figure 6 The first gathering drive motor 7-3 is connected to the first gathering plate 7-1 via a first coupling, and the second gathering drive motor 7-4 is connected to the second gathering plate 7-2 via a second coupling; a torque sensor 7-5 is provided on the first gathering plate 7-1 to obtain the torque during gathering; the first gathering drive motor 7-3 can be fixed on the harvesting frame 6, and the second gathering drive motor 7-4 can be fixed on the first gathering plate 7-1.

[0072] See Figure 6Both the first gathering drive motor 7-3 and the second gathering drive motor 7-4 are servo motors.

[0073] See Figures 1-3 and Figure 7 The lychee-shaped vibrating harvester also includes a control cabinet 9 and a power supply mounted on the transport mechanism 1. The power supply provides power to the lychee-shaped vibrating harvester. The navigation mechanism includes a first lidar 10 and a GPS 11. The first lidar 10 is mounted on the control cabinet 9 and provides high-precision environmental perception information. The GPS 11 is located at both ends of the control cabinet 9 and provides the position and speed information of the transport mechanism 1. By acquiring high-precision environmental perception information and the position and speed information of the transport mechanism 1, and integrating these data, comprehensive and accurate environmental perception and positioning navigation information is provided for the autonomous driving system. The control cabinet 9 can control the movement of the transport mechanism 1 through environmental perception and positioning navigation information, and can also control the harvesting device to perform harvesting actions.

[0074] See Figures 1-3 The multimodal sensing mechanism includes gimbal support frames 12 mounted on both sides of the movable support 2, a camera 14 mounted at the end of one of the gimbal support frames 12, and a second lidar 13 mounted at the end of the other gimbal support frame 12. The gimbal support frames 12 are mounted on the movable support 2 with screws. In the above structure, after calibration by the second lidar 13 and the camera 14, the three-dimensional point cloud information of the litchi tree canopy and the position information of the litchi fruit are obtained. After the harvesting device moves to the position of the litchi fruit to be harvested, the control cabinet 9 processes the three-dimensional point cloud information to obtain the three-dimensional information of the canopy near the litchi fruit to be harvested, calculates and judges the optimal harvesting angle, and converts the harvesting angle into an electrical signal to control the contouring mechanism 5 to adjust the contouring angle of the harvesting device.

[0075] See Figures 1-4 and Figure 8The contouring mechanism 5 includes a contouring support plate 5-1 vertically slidably mounted on the movable support 2 and an electric push rod 5-2 disposed between the contouring support plate 5-1 and the harvesting frame 6; the contouring support plate 5-1 is rotatably connected to the harvesting frame 6, one end of the electric push rod 5-2 is hinged to the contouring support plate 5-1, the other end of the electric push rod 5-2 is hinged to the harvesting frame 6, and the contouring support plate 5-1 is connected to the lifting mechanism 4. There are two electric push rods 5-2, which can improve the stability of the contouring mechanism 5. In the above structure, the contouring mechanism 5 realizes the tilt angle change of the harvesting device (0° to 120°) through the electric push rods 5-2. By extending and retracting the electric push rods 5-2, the harvesting frame 6 is pushed to swing, so that the angle between the contouring support plate 5-1 and the harvesting frame 6 changes, thereby changing the angle of the entire harvesting device. In the initial state, the harvesting frame 6 is at 90° with the ground, that is, it is parallel to the contouring support plate 5-1. When the electric push rods 5-2 extend, the harvesting frame 6 changes from 90° to 0°. When the electric push rods 5-2 retract, the harvesting frame 6 changes from 90° to 120°.

[0076] In terms of harvesting effect, a harvesting angle parallel to the lychee fruit will result in a better harvesting effect, that is, a larger actual harvesting area, a lower harvesting damage rate, and a higher harvesting cleanliness rate. The harvesting cleanliness rate is calculated as the number of lychee fruits in the fruit harvesting frame divided by the total number of lychee fruits in the harvesting area.

[0077] See Figure 8 The contoured support plate 5-1 is provided with two support seats 15, and the lower end of the harvesting frame 6 is provided with two fixed seats 16. A rotating shaft 17 is fixed on each of the two fixed seats 16, and the rotating shaft 17 is rotatably connected to the support seats 15. By setting the above structure, the rotatable connection between the contoured support plate 5-1 and the harvesting frame 6 is realized.

[0078] See Figures 1-3 The lychee contour-following vibrating harvester also includes a blower mechanism 18 mounted on the movable support 2; the blower mechanism 18 includes a fan mounting frame mounted on the movable support 2 and multiple blowers arranged side by side on the fan mounting frame. During the harvesting process, not only lychee fruits fall off when the fruit is tapped, but also lychee leaves. The blower mechanism 18 can blow away lychee leaves and small branches during the harvesting process, preventing lychee leaves and small branches from falling into the fruit collection mechanism.

[0079] See Figures 1-3 and Figure 8The translation mechanism 3 includes a slide rail 3-1 mounted on the top of the transport mechanism 1, a translation drive motor, and a translation lead screw and nut assembly; the moving bracket 2 is slidably mounted on the slide rail 3-1, the lead screw and nut of the translation lead screw and nut assembly are connected to the moving bracket 2, and the translation drive motor is connected to the lead screw of the translation lead screw and nut assembly; the lifting mechanism 4 includes a lifting drive motor 4-1 and a lifting lead screw and nut assembly; the lifting drive motor 4-1 is mounted on the upper end of the moving bracket 2, the lead screw and nut of the lifting lead screw and nut assembly are connected to the contour support plate 5-1, and the lead screw of the lifting lead screw and nut assembly is connected to the lifting drive motor 4-1. During harvesting, the translation drive motor drives the translation screw and nut pair to move, which in turn moves the moving support 2 horizontally, thereby causing the contouring mechanism 5 and the harvesting device to move horizontally together. The lifting drive motor 4-1 drives the lifting screw and nut pair to move, which in turn moves the contouring mechanism 5 and the harvesting device together vertically. Through the cooperation of the lifting mechanism 4, the translation mechanism 3 and the contouring mechanism 5, the harvesting device is ensured to perform harvesting operations at an appropriate height and distance.

[0080] Both the translation drive motor and the lifting drive motor 4-1 are stepper motors.

[0081] See Figures 1-3 The fruit collecting mechanism includes a lychee harvesting frame 19 mounted on the slide rail 3-1 and a guide enclosure 20 positioned between the harvesting device and the lychee harvesting frame 19. The guide enclosure 20 is connected to the movable support 2. In this structure, the guide enclosure 20 guides the lychee fruit to fall into the lychee harvesting frame 19. The lychee harvesting frame 19 can move horizontally synchronously with the movable support 2, ensuring that the horizontal positions of the lychee harvesting frame 19 and the guide enclosure 20 remain unchanged. This facilitates the lychee fruit falling into the lychee harvesting frame 19 under the guidance of the guide enclosure 20. The guide enclosure 20 can be connected to the movable support 2 via a screw drive element, which can drive the guide enclosure 20 to move vertically. Alternatively, the lifting screw nut pair can have two screw nuts, and the guide enclosure 20 can also be connected to one of the screw nuts in the lifting screw nut pair to achieve synchronous lifting and lowering movement of the guide enclosure 20 and the harvesting device.

[0082] Furthermore, a sliding seat can be provided between the lychee harvesting frame 19 and the slide rail 3-1. The lychee harvesting frame 19 and the sliding seat are detachably connected, the slide rail 3-1 is slidably connected to the sliding seat, and the sliding seat is connected to the movable support 2. When the movable support 2 moves on the slide rail 3-1, it will drive the sliding seat to move together, thereby realizing the synchronous horizontal movement of the lychee harvesting frame 19, the guide enclosure frame 20, and the harvesting device.

[0083] The guide frame 20 is a conical mesh structure, which is intended to facilitate the guidance of lychee fruits and the blowing mechanism 18.

[0084] See Figure 3 The transport mechanism 1 includes a chassis and tracks mounted on the chassis. The tracks can adapt to complex terrain, improving harvesting flexibility.

[0085] See Figures 1-4 The working principle of the above-mentioned lychee contour-following vibration harvesting machine is as follows:

[0086] The harvester moves to the designated position via a navigation mechanism. The lifting mechanism 4, translation mechanism 3, and contouring mechanism 5 drive the harvesting device to ensure it reaches the appropriate height, distance, and orientation (angle) for harvesting. A multimodal sensing mechanism identifies the litchi tree canopy and obtains information about the litchi fruits to be harvested. Based on this information, the contouring mechanism 5 adjusts the angle of the harvesting device to achieve the optimal harvesting posture. A gathering mechanism 7 gathers the litchi tree canopy branches, and then a vibration harvesting execution mechanism 8 taps the branches, causing them to vibrate and fall off, thus harvesting the litchi fruits. Finally, the fallen litchi fruits are collected by a fruit collection mechanism.

[0087] Example 2

[0088] See Figures 1-3 and Figure 9 This embodiment discloses a control method for a lychee contour-following vibration harvester as described in Embodiment 1, comprising the following steps:

[0089] (1) Positioning and navigation of navigation institutions:

[0090] The first lidar 10 acquires high-precision environmental perception information, and the GPS 11 acquires the position and speed information of the carrier 1. The acquired environmental perception information, position and speed information are used to locate the carrier 1, build a map and plan a path. The carrier 1 reaches the designated location according to the planned path.

[0091] (2) The lifting mechanism 4 and the translation mechanism 3 perform lifting and translation:

[0092] Depending on the different harvesting environment and lychee variety, the lifting mechanism 4 adjusts the height of the harvesting device, and the translation mechanism 3 adjusts the distance between the harvesting device and the lychee fruit to ensure that the harvesting device is operated at a suitable height and distance.

[0093] (3) Dataset creation:

[0094] Multiple images of the litchi tree canopy were acquired using camera 14 of the multimodal sensing mechanism; then, the acquired images of the litchi tree canopy were labeled using an annotation tool to select the litchi tree canopy and the litchi fruits to be picked in the canopy; thus, a litchi tree canopy and litchi fruit dataset was obtained.

[0095] (4) Constructing deep learning models:

[0096] The YOLO v9 network was used to train a litchi fruit dataset to obtain a network model that can identify the litchi tree canopy and litchi fruits.

[0097] (5) Joint calibration of the second lidar 13 and camera 14: The purpose of joint calibration of the second lidar 13 and camera 14 is to obtain the conversion relationship between the data of the second lidar 13 and camera 14, and to find the corresponding pixel points in the laser point cloud data and the litchi tree canopy image at the same time.

[0098] The specific principle is as follows: the lidar coordinate system can describe the relative position of an object and the lidar, represented as [X...]. L Y L Z L ], where the origin is the geometric center of the lidar, X L The axis is horizontal forward, Y L The axis is horizontal to the left, Z L The axis points vertically upwards, conforming to the right-hand coordinate system rule. The camera system has three coordinate systems: pixel coordinates, image coordinates, and camera coordinates. The pixel coordinates are represented as [u, v], where the origin is the upper left corner of the image, the u-axis is horizontal to the right, and the v-axis is vertically downwards. The image coordinates are established below the pixel coordinates, using physical units (such as millimeters) to give the pixel scale physical meaning; they are represented as [x, y]. The origin is the camera principal point, the intersection of the camera's principal optical axis and the imaging plane, generally located at the center of the imaging plane. The x-axis is parallel to the u-axis, and the y-axis is parallel to the v-axis. The camera coordinates describe the relative position of an object to the camera, represented as [X...]. C Y C Z C ], where the origin is point O, the optical center of camera 14, X C The axis is parallel to the x-axis, and the y-axis is parallel to the x-axis. C The Z-axis is parallel to the y-axis. C The axis is parallel to the camera's principal optical axis and perpendicular to the image plane. The specific transformation process is as follows: Figure 11 As shown.

[0099] See Figure 11 The specific steps for the 14-coordinate transformation between the lidar and the camera are as follows:

[0100] (5.1) Transform from the lidar coordinate system to the camera coordinate system; represented by the rotation matrix R and the translation matrix T. Where R is a 3×3 matrix representing spatial coordinate rotation, and T is a 3×1 matrix representing spatial coordinate translation; the specific formula is as follows:

[0101]

[0102] (5.2) The transformation from the camera coordinate system to the image coordinate system is the process of projecting a three-dimensional object into a two-dimensional graphic. This belongs to perspective projection and satisfies the similarity theorem of triangles. Where f is the camera's focal length (14), the specific formula is as follows:

[0103]

[0104] (5.3) Transformation from image coordinate system to pixel coordinate system: There is no rotation transformation, but the origin position and unit length are different. It mainly involves scaling and translation transformations. The specific formulas are as follows:

[0105]

[0106] In summary, the coordinate transformation relationship between the lidar and camera 14 can be expressed as:

[0107]

[0108] The above formula can be used to project laser point cloud data onto an image (lychee tree canopy image). The coordinate transformation relationship can be obtained through Zhang Zhengyou's calibration method, thus realizing the joint spatial calibration of the second lidar 13 and camera 14.

[0109] (6) Time registration of the second lidar 13 and camera 14:

[0110] Since the acquisition frequency of camera 14 is higher than that of the second lidar 13, and the image data of camera 14 and the laser point cloud data of the second lidar 13 have synchronized timestamps, the time nearest neighbor matching method is used to find the image data with the smallest time interval with each frame of the second lidar 13 data and process it to achieve time registration between the second lidar 13 and camera 14.

[0111] The point cloud coordinates are transformed into the camera 14 world coordinate system through the joint calibration extrinsic transformation matrix. Then, according to the perspective imaging model, the coordinates are projected into the pixel coordinate system using the intrinsic parameter matrix obtained by camera 14 calibration to obtain the corresponding pixel coordinates, thereby realizing the registration of 3D point cloud and 2D image.

[0112] (7) Three-dimensional reconstruction:

[0113] The color image of the litchi fruit acquired by camera 14 is input into the network model that has been trained in step (4) to obtain the location coordinate information of the litchi fruit picking, and the three-dimensional point cloud of the litchi fruit is obtained according to the calibration and registration results of steps (5) and (6).

[0114] (8) Semantic segmentation for 3D data:

[0115] The direct processing method is adopted, using the Point-Net network to transform the 3D point cloud obtained in step (7) into a regular 3D voxel mesh or image set. The Point-Net network performs feature extraction through learning, extracting local and global features from the original 3D point cloud data, and capturing the semantic relationships and spatial structure between different categories. The original 3D point cloud data is the 3D point cloud data obtained in step (7).

[0116] Then, the Point-Net network outputs the semantic category prediction for each point or voxel, which is then assigned to the corresponding semantic labels for the fruit tree canopy and lychee fruit, thus achieving semantic segmentation of the 3D point cloud data.

[0117] Next, the cross-entropy loss function is used to measure the difference between the predicted result and the true label. The backpropagation algorithm and Adam optimizer are used to adjust the network parameters and minimize the loss function, so that the network can accurately predict the semantic category of each point or voxel.

[0118] 3D point clouds are 3D data. The above steps better preserve spatial information and details when processing 3D data, which helps to accurately understand and infer semantic information in the 3D environment. This allows for the semantic segmentation of litchi tree canopies and litchi fruits in 3D data, and the network parameters are progressively optimized during training to obtain more accurate semantic segmentation results for litchi tree canopies and litchi fruits.

[0119] Point-Net is a neural network architecture for processing point cloud data, capable of directly manipulating unordered point sets and learning global features.

[0120] The cross-entropy loss function is a commonly used loss function in machine learning and deep learning, particularly suitable for classification problems. It measures the difference between the model's predictions and the true labels, that is, the difference between the probability distribution of the model's output and the discrete distribution of the actual labels.

[0121] Backpropagation is one of the key algorithms used in deep learning to train neural networks. It calculates the gradient of the loss function with respect to each parameter, then updates the parameters in the reverse direction of the gradient, thereby continuously optimizing the neural network and gradually improving the model's performance.

[0122] The Adam optimizer (Adaptive Moment Estimation) is a commonly used stochastic optimization algorithm that combines the characteristics of momentum methods and adaptive learning rates, and is widely used in the training of deep learning models. The Adam optimizer can dynamically adjust the learning rate based on different parameter updates, thereby more effectively optimizing the model's convergence process.

[0123] (9) Motion trajectory planning:

[0124] The semantically segmented 3D point cloud of the litchi fruit is transmitted to the control cabinet 9. The control cabinet 9 analyzes the spatial 3D information contained in the 3D point cloud and uses an obstacle avoidance algorithm to plan the motion trajectory of the litchi contour vibrating harvester so that it gets close to the harvesting target. After semantic segmentation, the coordinates of the litchi fruit cluster center are output. The translation mechanism 3 and the lifting mechanism 4 in the lifting and translation contour mechanism are controlled by a stepper motor so that the harvesting device gets close to the coordinates of the litchi fruit cluster center.

[0125] (10) Harvesting posture adjustment:

[0126] The direction vector n of the lychee fruit clustering axis is determined by acquiring the 3D point cloud of the lychee fruit from semantic segmentation; the direction vector n is then transmitted to the contouring mechanism 5, and contouring is performed from the direction of the direction vector n.

[0127] Fruit clustering refers to the process of semantically segmenting litchi fruits, dividing a group of litchi fruits into a harvesting area, and then selecting the most suitable angle for harvesting this group of litchi fruits through decision-making. That is, in the 3D point cloud information of the semantically segmented litchi fruits, a point is selected as the harvesting center point (i.e., the coordinates of the litchi fruit cluster center), and the direction vector n most suitable for the harvesting angle is selected based on the canopy information of the harvesting center point. The litchi fruit clustering axis is the axis that passes through the harvesting center point and is parallel to the cross-section on the canopy at that point.

[0128] (11) Harvesting and Collection:

[0129] By controlling the closing mechanism 7 to close and surround the picking target, the vibration harvesting execution mechanism 8 vibrates and harvests the picking target from the direction vector n, and the harvested lychee fruits fall into the fruit collection mechanism for collection.

[0130] See Figure 10 The harvesting process of the lychee contouring vibration harvester in this embodiment is as follows:

[0131] The machine is initialized. This machine is a lychee contour-following vibrating harvester. First, it checks whether the lychee harvesting frame 19 is full. If it is full, the lychee harvesting frame 19 needs to be replaced. The lychee contour-following vibrating harvester autonomously navigates to the designated location. The multimodal sensing mechanism collects images. The control cabinet 9 plans the motion trajectory based on the collected images and determines the optimal harvesting direction vector. Based on the planned motion trajectory and the optimal harvesting direction vector, the contour-following vibrating harvester achieves contour-following vibrating harvesting and collects the fruits. If all the fruits are harvested, the harvesting ends.

[0132] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A lychee contour-following vibrating harvesting machine, characterized in that, The system includes a transport mechanism for movement, a navigation mechanism mounted on the transport mechanism for positioning and navigation, a multimodal sensing mechanism for identifying the litchi tree canopy, a harvesting device for harvesting litchi fruits, a fruit collecting mechanism for collecting litchi fruits, and a lifting, translating, and contour-following mechanism for driving the harvesting device. The multimodal sensing mechanism identifies the litchi tree canopy and obtains information about the litchi fruits to be harvested within it. Based on the litchi fruit information, the contour-following mechanism adjusts the angle of the harvesting device. The lifting and translating contouring mechanism includes a movable support, a translation mechanism, a lifting mechanism, and a contouring mechanism; wherein, the movable support is horizontally slidably disposed on the top of the transport mechanism; the contouring mechanism is vertically slidably disposed on the movable support and is used to adjust the harvesting angle of the harvesting device; the lifting mechanism is disposed on the movable support and is used to drive the contouring mechanism to perform vertical movement to drive the harvesting device to perform vertical movement; the translation mechanism is used to drive the movable support to perform horizontal movement to drive the harvesting device to perform horizontal movement. The harvesting device includes a harvesting frame mounted on the contouring mechanism, a gathering mechanism mounted on the harvesting frame for gathering the canopy of the litchi tree, and a vibrating harvesting execution mechanism; wherein, the multimodal sensing mechanism is mounted on the movable support, and the fruit collecting mechanism is located below the vibrating harvesting execution mechanism. The vibratory harvesting actuator includes a vibratory tapping mechanism mounted on the harvesting frame and a crank-connecting rod drive mechanism for driving the vibratory tapping mechanism; wherein, the vibratory tapping mechanism includes vibratory tapping assemblies mounted on both sides of the harvesting frame, the vibratory tapping assemblies include harvesting rollers rotatably mounted on the harvesting frame and multiple tapping rods mounted on the harvesting rollers, and the crank-connecting rod drive mechanism is connected to the two harvesting rollers; The crank-connecting rod drive mechanism includes a vibratory tapping drive motor, a crank, a first intermediate connecting rod, a first rocker arm, a second intermediate connecting rod, and a second rocker arm. The vibratory tapping drive motor is located at the lower end of the harvesting frame. One end of the crank is connected to the main shaft of the vibratory tapping drive motor. One end of the first intermediate connecting rod and one end of the second intermediate connecting rod are connected to the other end of the crank. One end of the first rocker arm is connected to the other end of the first intermediate connecting rod, and the other end of the first rocker arm is connected to one of the harvesting rollers. One end of the second rocker arm is connected to the other end of the second intermediate connecting rod, and the other end of the second rocker arm is connected to another harvesting roller. The first intermediate connecting rod and the second intermediate connecting rod have different lengths, and the first rocker arm and the second rocker arm have different lengths. The gathering mechanism includes two gathering arms disposed opposite to each other on the harvesting frame; each gathering arm includes a first gathering plate, a second gathering plate, a first gathering drive motor, and a second gathering drive motor; the first gathering plate is disposed between the harvesting frame and the second gathering plate, one end of the first gathering plate is rotatably connected to the harvesting frame, and the other end is rotatably connected to the end of the second gathering plate; the first gathering drive motor is used to drive the first gathering plate to rotate on the harvesting frame, and the second gathering drive motor is used to drive the second gathering plate to rotate on the first gathering plate.

2. The litchi contour-following vibration harvesting machine according to claim 1, characterized in that, The lychee contouring vibration harvester also includes a blower mechanism mounted on the movable support; the blower mechanism includes a blower mounting frame mounted on the movable support and multiple blowers arranged side by side on the blower mounting frame.

3. The litchi contour-following vibrating harvester according to claim 1, characterized in that, The lychee contouring vibration harvester also includes a control cabinet mounted on the transport mechanism. The navigation mechanism includes a first laser radar and a GPS. The first laser radar is mounted on the control cabinet to provide high-precision environmental perception information. The GPS is located at both ends of the control cabinet to provide the position and speed information of the transport mechanism.

4. A lychee contour-following vibrating harvester according to claim 3, characterized in that, The multimodal sensing mechanism includes gimbal support frames mounted on both sides of the movable support, a camera mounted at the end of one of the gimbal support frames, and a second lidar mounted at the end of the other gimbal support frame.

5. A lychee contour-following vibrating harvester according to claim 4, characterized in that, The contouring mechanism includes a contouring support plate vertically slidably mounted on the movable support and an electric push rod disposed between the contouring support plate and the harvesting frame; the contouring support plate is rotatably connected to the harvesting frame, one end of the electric push rod is hinged to the contouring support plate, the other end of the electric push rod is hinged to the harvesting frame, and the contouring support plate is connected to the lifting mechanism.

6. A lychee contour-following vibrating harvester according to claim 5, characterized in that, The translation mechanism includes a slide rail, a translation drive motor, and a translation screw and nut assembly mounted on the top of the transport mechanism; the moving bracket is slidably mounted on the slide rail, the screw and nut of the translation screw and nut assembly are connected to the moving bracket, and the translation drive motor is connected to the screw of the translation screw and nut assembly; the lifting mechanism includes a lifting drive motor and a lifting screw and nut assembly; the lifting drive motor is mounted on the upper end of the moving bracket, the screw and nut of the lifting screw and nut assembly are connected to the contour support plate, and the screw of the lifting screw and nut assembly is connected to the lifting drive motor.

7. A lychee contour-following vibrating harvester according to claim 6, characterized in that, The fruit collection mechanism includes a lychee harvesting frame mounted on the slide rail and a guide enclosure frame mounted between the harvesting device and the lychee harvesting frame, the guide enclosure frame being connected to the movable support.

8. A control method for a litchi contour-following vibrating harvester as described in any one of claims 4-7, characterized in that, Includes the following steps: (1) Positioning and navigation of navigation agencies: The first lidar acquires high-precision environmental perception information, and the GPS acquires the position and speed information of the vehicle. The acquired environmental perception information, position and speed information are used to locate the vehicle, build a map and plan a path. The vehicle then reaches the designated location according to the planned path. (2) The lifting mechanism and the translation mechanism perform lifting and translation: The lifting mechanism adjusts the height of the harvesting device, and the translation mechanism adjusts the distance between the harvesting device and the lychee fruit, ensuring that the harvesting device can be used for harvesting operations at a suitable height and distance. (3) Dataset creation: Multiple images of the litchi tree canopy were acquired using a camera from a multimodal sensing mechanism. Then, the acquired images of the litchi tree canopy were labeled using a labeling tool to select the litchi tree canopy and the litchi fruits to be picked in the canopy. This resulted in a litchi tree canopy and litchi fruit dataset. (4) Constructing a deep learning model: The YOLO v9 network was used to train the litchi fruit dataset to obtain a network model that can identify the litchi tree canopy and litchi fruit. (5) Joint calibration of the second lidar and camera to obtain the conversion relationship between the second lidar and camera data, and find the corresponding pixel points in the lidar point cloud data and the litchi tree canopy image at the same time; (6) Time registration of the second lidar and camera: The image data with the smallest time interval with each frame of the second lidar data is found by using the time nearest neighbor matching method and then processed to achieve time registration between the second lidar and the camera. (7) Three-dimensional reconstruction: Input the color image of the litchi fruit acquired by the camera into the network model that has been trained in step (4) to obtain the location coordinate information of the litchi fruit picking, and obtain the three-dimensional point cloud of the litchi fruit according to the calibration and registration results of steps (5) and (6). (8) Semantic segmentation for 3D data: The direct processing method is adopted, and the Point-Net network is used to transform the three-dimensional point cloud obtained in step (7) into a regular 3D voxel mesh or image set. The Point-Net network performs feature extraction through learning, extracts local and global features from the original three-dimensional point cloud data, and captures the semantic relationship and spatial structure between different categories. Then, the Point-Net network outputs the semantic category prediction for each point or voxel, and assigns it to the corresponding semantic labels of the fruit tree canopy and lychee fruit to achieve semantic segmentation of the 3D point cloud data. (9) Motion trajectory planning: The semantically segmented 3D point cloud of the lychee fruit is transmitted to the control cabinet. The control cabinet analyzes the spatial 3D information contained in the 3D point cloud and uses an obstacle avoidance algorithm to plan the motion trajectory of the lychee contour vibrating harvester so that it can get closer to the harvesting target. (10) Harvesting posture adjustment: The 3D point cloud of the litchi fruit obtained by semantic segmentation is used to determine the direction vector n of the litchi fruit cluster axis; the direction vector n is transmitted to the contouring mechanism, and contouring is performed from the direction of the direction vector n. (11) Harvesting and Collection: The harvesting target is surrounded by a closing mechanism, and the vibrating harvesting mechanism harvests the target by vibrating in the direction of direction vector n. The harvested lychee fruits fall into the fruit collection mechanism for collection.

Images