Mechanical arm vibration auxiliary automatic sea-buckthorn harvesting device based on machine vision and grabbing point positioning and sea-buckthorn harvesting method

Abstract

The invention discloses a mechanical arm vibration auxiliary automatic sea-buckthorn harvesting device based on machine vision and a grabbing point positioning and sea-buckthorn harvesting method, relates to the technical field of sea-buckthorn harvesting, and solves the problem that sea-buckthorn harvesting is difficult. A crawler belt moving device is provided with a rubber crawler belt, a picking mechanical arm has six degrees of freedom, and the mechanical arm has six degrees of freedom; the tail end execution device is sent to a harvesting position in a point-to-point mode, the tail end execution device vibrates branches, close to the tail end, of a sea-buckthorn tree and detects the vibration amplitude of the branches of the sea-buckthorn tree, then comparison of a binocular vision system is combined, an excitation power source is combined to output an electric signal with the appropriate frequency, the tail end execution device has the optimal amplitude, and the recovery efficiency is improved; the collecting umbrella device works cooperatively and is unfolded when the tail end executing device works, fallen fruits are collected, the fallen fruits fall into a fruit frame of the fallen fruit collecting device, the fruit frame is placed on a pressure sensor, workers are informed to timely come to replace the fruit frame, and efficient and continuous harvesting work of the sea-buckthorn trees planted in a plantation in a large scale is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of seabuckthorn harvesting, and in particular to a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, and a method for grasping point positioning and seabuckthorn harvesting. Background Technique

[0002] Seabuckthorn is a deciduous shrub or small tree, with a general plant height of about 2.5 meters. Its bark is relatively rough, and there are many thorns on the branches. Its fruit, the seabuckthorn fruit, is a berry, spherical or oval in shape, with a diameter of about 4-6 mm, and has rich nutritional and medicinal values, and is an important cash crop. Seabuckthorn has good cold tolerance, and areas with an average annual temperature of -5°C to 10°C are more suitable for seabuckthorn planting. Moreover, the ripening of seabuckthorn fruits is concentrated in September to October. After the fruits ripen, they need to be harvested in a concentrated and timely manner, otherwise it will seriously affect the quality and value of seabuckthorn fruits.

[0003] At present, there is little research on equipment in the field of seabuckthorn harvesting. Seabuckthorn picking is mainly manual picking or semi-automatic picking by combining manual labor with some handheld devices such as handheld fruit shakers. Since seabuckthorn fruits grow in clusters, are small and dense; and seabuckthorn trees have many hard thorns; and the temperature is relatively low during the ripening season, these situations bring great difficulties to traditional seabuckthorn harvesting methods, seriously affecting the harvesting efficiency, and huge waste is caused every year due to incomplete harvesting. Summary of the Invention

[0004] Aiming at the problem of difficult seabuckthorn harvesting mentioned above, the purpose of the present invention is to provide a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, and a method for grasping point positioning and seabuckthorn harvesting. Based on the currently widely used vibration harvesting method, it can realize automatic seabuckthorn harvesting, improve the harvesting efficiency, reduce the labor force, increase the yield of the plantation, and improve the benefits.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, which includes: a crawler mobile device 1, a control cabinet 2, a picking mechanical arm 3, an end effector 4, a binocular vision system 5, a collection umbrella device 6 and a fallen fruit collection device 7. The control cabinet 2 is installed on the crawler mobile device 1, the end effector 4 is installed on the crawler mobile device 1 through the picking mechanical arm 3, the picking mechanical arm 3 is arranged in front of the control cabinet 2, the fallen fruit collection device 7 is connected to the crawler mobile device 1, both the binocular vision system 5 and the collection umbrella device 6 are installed on the fallen fruit collection device 7, and the collection umbrella device 6 is arranged in front of the binocular vision system 5;

[0007] The crawler moving device 1 is used to realize the forward movement, backward movement and turning of the seabuckthorn harvesting device. The binocular vision system 5 is used to detect the positions of the seabuckthorn tree 801 and the seabuckthorn fruits. The end effector 4 is used to grasp the seabuckthorn tree 801 and cause the seabuckthorn fruits to fall by vibrating the seabuckthorn tree 801. The collection umbrella device 6 is used to receive the falling seabuckthorn fruits and transfer them into the fruit collection device 7. The picking robotic arm 3 is used to adjust the working position and angle of the end effector 4. The control cabinet 2 is used to realize the automatic control of the seabuckthorn harvesting device.

[0008] For the above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, the crawler moving device 1 includes: a vehicle frame 103, a left-side drive traveling mechanism, a right-side drive traveling mechanism and a mounting plate 106. The left-side drive traveling mechanism and the right-side drive traveling mechanism are respectively installed on the left and right sides of the vehicle frame 103 and are driven independently, and the mounting plate 106 is installed on the top of the first vehicle frame 103.

[0009] For the above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, both the left-side drive traveling mechanism and the right-side drive traveling mechanism are crawler drive traveling mechanisms.

[0010] For the above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, the left-side drive traveling mechanism includes: a left-side stepping motor 104 and a left-side rubber crawler 105, and the left-side stepping motor 104 is used to drive the left-side rubber crawler 105 to travel; the right-side drive traveling mechanism includes: a right-side rubber crawler 101 and a right-side stepping motor 102, and the right-side stepping motor 102 is used to drive the right-side rubber crawler 101 to travel.

[0011] For the above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, the control cabinet 2 includes: a sheet metal cover 201, a PLC 202, an embedded processor 203, a vibration excitation power supply 204 and a gasoline generator 205. The sheet metal cover 201 is installed on the upper surface of the mounting plate 106, and the PLC 202, the embedded processor 203, the vibration excitation power supply 204 and the gasoline generator 205 are all arranged inside the sheet metal cover 201.

[0012] The above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, wherein the end effector 4 includes: a mechanical claw 401, an acceleration sensor 402, a lead screw nut 403, a polarization motor 404, a lead screw motor 405, and a rectangular sheet metal cover 406. The mechanical claw 401 is used to grasp the seabuckthorn tree 801, and the lead screw motor 405 is used to drive the mechanical claw 401 to perform the action of grasping or releasing. The acceleration sensor 402 is installed on the mechanical claw 401, and both the polarization motor 404 and the mechanical claw 401 are installed on the rectangular sheet metal cover 406. The polarization motor 404 is arranged below the lead screw motor 405, and the polarization motor 404 is used to drive the mechanical claw 401 to vibrate.

[0013] The mechanical claw 401 includes: a top link 4011, a limit link 4012, a transmission link 4013, and a drive link 4014. The lead screw nut 403 includes: a lead screw and a nut, and the nut is installed on the lead screw. The mechanical claws 401 are symmetrically arranged. The rear ends of both drive links 4014 are rotatably connected to the outer circumference of the nut. The rear end of each transmission link 4013 is rotatably connected to the front end of a drive link 4014. The front end of each transmission link 4013 is rotatably connected to the rear end of a top link 4011. The rear end of each limit link 4012 is rotatably connected to the rectangular sheet metal cover 406, and the front end of each limit link 4012 is rotatably connected to the middle of a top link 4011.

[0014] The above-mentioned robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, wherein the fruit drop collection device 7 includes: a proximity sensor 701, a collection vehicle 702, a fruit basket 703, a pressure sensor 704, and a baffle 705. The rear end of the collection vehicle 702 is connected to the front end of the vehicle frame 103. The collection vehicle 702 is a frame structure with an open top. The inside of the collection vehicle 702 is used to place the fruit basket 703. Baffles 705 are installed on both the left and right sides of the collection vehicle 702, and both baffles 705 are used to limit the fruit basket 703. The pressure sensor 704 is installed inside the collection vehicle 702, and the pressure sensor 704 is used to detect the mass of the fruit basket 703. The proximity sensor 701 is installed on the front side of the collection vehicle 702, and the proximity sensor 701 is used to detect the distance between the seabuckthorn tree 801 and the collection vehicle 702.

[0015] The above-mentioned mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, wherein the collection umbrella device 6 includes: a collection umbrella motor 601, an umbrella frame 602 and an umbrella cloth 603. The umbrella frame 602 includes a U-shaped base and a plurality of umbrella rods. The U-shaped base is installed in the middle of the front side of the top of the collection vehicle 702. The plurality of umbrella rods are arranged along the upper surface of the U-shaped base. The plurality of umbrella rods are rotatably installed on the U-shaped base. Two collection umbrella motors 601 are respectively installed at both ends of the U-shaped base. The two collection umbrella motors 601 are respectively used to drive the umbrella rods at both ends of the U-shaped base to rotate. The plurality of umbrella rods all penetrate into the umbrella cloth 603. By driving the umbrella rods at both ends of the U-shaped base to rotate through the collection umbrella motor 601, the umbrella cloth 603 is surrounded by the seabuckthorn tree 801 and the umbrella cloth 603 is retracted.

[0016] The above-mentioned mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, wherein the binocular vision system 5 includes: a first binocular camera 501 and a second binocular camera 502. The first binocular camera 501 and the second binocular camera 502 are respectively installed at the left and right ends of the rear side of the top of the collection vehicle 702. Both the first binocular camera 501 and the second binocular camera 502 are higher than the upper surface of the umbrella cloth 603, and the viewing directions of the first binocular camera 501 and the second binocular camera 502 are adjustable.

[0017] The above-mentioned mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, wherein the picking mechanical arm 3 includes: a bottom support rod 31, a middle connecting rod 32, a top connecting rod 33 and a top connecting flange 34. The bottom support rod 31 is perpendicular to the upper surface of the mounting plate 106. The lower end of the bottom support rod 31 and the mounting plate 106 are rotatably connected through a rotating seat. The lower end of the middle connecting rod 32 and the upper end of the bottom support rod 31 are rotatably connected through a rotating shaft. A slide rail is provided on the side surface of the top connecting rod 33. The rotating shaft seat is rotatably installed at the top of the middle connecting rod 32. The rotating shaft seat is slidably connected to the slide rail of the top connecting rod 33. The top connecting flange 34 is rotatably installed at the top of the top connecting rod 33 through a rotating shaft. The end effector 4 is rotatably installed on the top connecting flange 34 through a rotating shaft;

[0018] The picking robotic arm 3 further includes: a first drive motor 35, a second drive motor 36, a third drive motor 37, a fourth drive motor, a fifth drive motor 38, and a sixth drive motor 39. The first drive motor 35 is used to drive the rotating base to operate, enabling the bottom support rod 31 to rotate around its own axis. The second drive motor 36 is used to drive the lower end of the middle connecting rod 32 to rotate around the rotating shaft. The third drive motor 37 is used to drive the top connecting rod 33 to rotate around the rotating shaft seat. The fourth drive motor is used to drive the rotating shaft seat to slide along the slide rail of the top connecting rod 33. The fifth drive motor 38 is used to drive the top connecting flange 34 to pitch and rotate around the top end of the top connecting rod 33. The sixth drive motor 39 is used to drive the end effector 4 to rotate around its own axis on the top connecting flange 34.

[0019] Due to the adoption of the above technologies, the positive effects of the present invention compared with the prior art are as follows:

[0020] (1) The present invention realizes the efficient and continuous harvesting of sea buckthorn trees planted on a large scale in plantations; the crawler mobile device is equipped with rubber crawlers, enabling the harvesting device to move freely in the loam environment of the plantation, ensuring the continuity of the harvesting operation. At the same time, using the differential steering function, the device can reach any designated position; the picking robotic arm has six degrees of freedom and can avoid obstacles according to the program, and send the end effector point-to-point to the harvesting position; the end effector vibrates the near-end branches of the sea buckthorn tree. Compared with the traditional method of shaking the tree trunk for harvesting, it has a better fruit drop efficiency. At the same time, vibrating the end branches will not damage the tree trunk and roots, will not affect the vitality of the sea buckthorn tree, and will not have a negative impact on the fruit yield in the second year; and the end effector is equipped with an acceleration sensor to detect the vibration amplitude of the sea buckthorn branches, and combines with the excitation power supply to output an electrical signal with a suitable frequency to make the end effector have an optimal amplitude, which can significantly improve the fruit drop rate. Then, combined with the comparison of the binocular vision system, the remaining sea buckthorn fruits on the branches can be detected. If there are more remaining sea buckthorn fruits, the amplitude of the electrical signal output by the power supply will be further increased, so that the amplitude of the end effector is further increased, and the remaining sea buckthorn fruits are shaken off to improve the harvesting rate; the collection umbrella device works in coordination, follows the production rhythm, unfolds when the end effector is working, collects the fallen fruits, and makes them fall into the fruit box of the fallen fruit collection device. The fruit box is placed on the pressure sensor. Before the weight of the fruit box reaches the set value, the harvesting device will keep working, further improving the harvesting efficiency; after the weight of the fruit box reaches the set value, the harvesting device stops in time, sends a signal to the control center, notifies the worker to come and replace the fruit box, and can continue to work after restarting after replacement.

[0021] (2) The various devices of the present invention work in coordination, and on the basis of low-intensity work with less labor, can realize the automatic, continuous, and efficient harvesting operation of sea buckthorn trees in the plantation. Description of the Drawings

[0022] Figure 1 It is a schematic diagram of the overall structure of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 2 It is a schematic diagram of the structure of the crawler moving device and the control box of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 3 It is a schematic diagram of the structure of the picking mechanical arm of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 4 It is a schematic diagram of the structure of the end effector of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 5 It is a schematic diagram of the structure of the collection umbrella device of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 6 It is a schematic diagram of the structure of the fruit drop collection device of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 7 It is a schematic diagram of the working path of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 8 It is a schematic diagram of the working state of the binocular vision system of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision in the vertical direction according to the present invention. Figure 9 It is a schematic diagram of the working state of the binocular vision system of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision in the horizontal direction according to the present invention. Figure 10 It is the overall working flow chart of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 11 It is the machine vision-based positioning flow chart of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 12 It is a schematic diagram of the seabuckthorn tree positioning of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention. Figure 13 It is a schematic diagram of the seabuckthorn fruit positioning of a mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to the present invention.

[0023] In the accompanying drawings: 1. Crawler moving device; 2. Control cabinet; 3. Harvesting robotic arm; 4. End effector; 5. Binocular vision system; 6. Collection umbrella device; 7. Fruit drop collection device; 31. Bottom support rod; 32. Middle connecting rod; 33. Top connecting rod; 34. Top connecting flange; 35. First drive motor; 36. Second drive motor; 37. Third drive motor; 38. Fifth drive motor; 39. Sixth drive motor; 101. Right rubber crawler; 102. Right stepping motor; 103. Frame; 104. Left stepping motor; 105. Left rubber crawler; 106. Mounting plate; 201. Sheet metal cover; 202. PLC; 203. Embedded processor; 204. Vibration excitation power supply; 205. Gasoline generator; 401. Mechanical claw; 402. Acceleration sensor; 403. Lead screw nut; 404. Polarization motor; 405. Lead screw motor; 406. Cuboid sheet metal cover; 501. First binocular camera; 502. Second binocular camera; 601. Collection umbrella motor; 602. Umbrella frame; 603. Umbrella cloth; 701. Proximity sensor; 702. Collection vehicle; 703. Fruit box; 704. Pressure sensor; 705. Baffle; 801. Sea buckthorn tree; 802. Starting position of the harvesting device; 803. Harvesting position of the harvesting device; 804. Pause position of the harvesting device. Detailed implementation manners

[0024] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but it is not intended to limit the present invention.

[0025] Please refer to Figures 1 to 13 As shown, a mechanical arm vibration-assisted automatic sea buckthorn harvesting device based on machine vision is shown. Combining with the actual requirements of current sea buckthorn harvesting operations, to achieve automatic sea buckthorn harvesting, reduce the difficulty of harvesting operations, and improve the harvesting efficiency, it includes: a crawler moving device 1, a control cabinet 2, a harvesting robotic arm 3, an end effector 4, a binocular vision system 5, a collection umbrella device 6, and a fruit drop collection device 7, a total of seven parts. Among them, the control cabinet 2 and the harvesting robotic arm 3 are fixed on the crawler moving device 1 by bolts, the end effector 4 is fixed at the end of the harvesting robotic arm 3 by screws, the collection umbrella device 6 is connected to the fruit drop collection device 7 by a snap ring, and then the crawler moving device 1 and the fruit drop collection device 7 are connected by a connecting plate and bolts.

[0026] Among them, the crawler moving device 1 mainly includes two crawlers and two stepping motors. The crawler moving device 1 serves as the main power of the entire device. The crawlers are selected to facilitate movement on soft ground. At the same time, by controlling the speed difference between the two stepping motors by the controller, the crawler moving device 1 can move in a straight line or turn. Coupled with the binocular vision system 5, a closed loop can be achieved, enabling the harvesting device to move forward along the planned route.

[0027] Further, in the control cabinet 2, there are a gasoline generator 205, an embedded processor 203, a PLC 202, and a vibration excitation power supply 204. Among them, the gasoline generator 205 provides power for the entire seabuckthorn harvesting device, increasing the endurance and improving the efficiency. The embedded processor 203 is responsible for processing the data from the binocular vision system 5 and sending the processed data to the PLC 202 to work in coordination with the PLC 202. The PLC 202 is responsible for controlling the actions of all the motors on the harvesting device, receiving the data from the embedded processor 203, and can directly control the execution of the crawler moving device 1, the picking robotic arm 3, and the end effector 4 to ensure the smoothness of the harvesting rhythm. The vibration excitation power supply 204 is connected to the gasoline generator 205 to directly supply power to all the electrical appliances on the harvesting device. At the same time, it can control the working condition of the end effector 4 by outputting electrical signals with different frequencies and amplitudes.

[0028] Further, the picking robotic arm 3 has six degrees of freedom. When harvesting seabuckthorn fruits, it can bend freely to avoid obstacles and accurately send the end effector 4 to the calculated grasping point.

[0029] Further, the end effector 4 includes a mechanical claw 401, a lead screw nut 403, a polarization motor 404, and an acceleration sensor 402. Among them, the nut of the lead screw nut 403 is connected to the bottom link of the linkage mechanism of the mechanical claw 401, and the lead screw of the lead screw nut 403 is connected to the lead screw motor 405 through a coupling. The rotation of the lead screw motor 405 controls the clamping and loosening of the mechanical claw 401. The polarization motor 404 can vibrate the mechanical claw 401, and then drive the branches clamped by the mechanical claw 401 to vibrate to achieve fruit dropping. The polarization motor 404 is controlled by the vibration excitation power supply 204 and can achieve different amplitudes and frequencies. The acceleration sensor 402 can detect these data of the target object and then feedback them to the controller to achieve dynamic regulation of the amplitude and frequency.

[0030] Further, the binocular vision system 5 includes two binocular cameras with image acquisition cards. Because the branches of the seabuckthorn tree 801 are numerous and dense, different layout methods of the two binocular cameras can expand the field of view, reduce the blind area, increase the baseline length at the same time, expand the effective range of depth measurement, and improve the accuracy of depth measurement.

[0031] Further, the collection umbrella device 6 includes an umbrella frame 602, an umbrella cloth 603, and two collection umbrella motors 601. The umbrella cloth 603 is a nylon cloth with elastic bands at the edges and openings at corresponding positions, and is worn on the umbrella frame 602. The two ends of the bottom of the umbrella frame 602 are connected to the collection umbrella motors 601. Controlled by the PLC 202, the collection umbrella motors 601 rotate to drive the umbrella cloth 603 to unfold, and can achieve surrounding and clinging to the seabuckthorn tree 801.

[0032] Furthermore, the fallen fruit collection device 7 includes a collection vehicle 702, a fruit frame 703, a proximity sensor 701, and a pressure sensor 704. The collection vehicle 702 is a frame structure, and the fruit frame 703 is placed in the middle. When harvesting, the sea buckthorn fruits fall from the holes of the umbrella cloth 603 into the fruit frame 703, and a pressure sensor 704 is installed below the fruit frame 703 to detect the weight of the collected fallen fruits. The proximity sensor 701 is installed in the circular groove in front of the collection vehicle 702 to detect the distance between the fallen fruit collection device 7 and the sea buckthorn tree 801.

[0033] Further, in a preferred embodiment, the end effector 4 includes: a mechanical claw 401, an acceleration sensor 402, a lead screw nut 403, a polarization motor 404, a lead screw motor 405 and a rectangular sheet metal cover 406, the mechanical claw 401 is a connecting rod mechanism formed by hinged connection of several connecting rods, wherein the top connecting rod grasping surface is arc-shaped, used to realize the grasping function, and the other connecting rods are ordinary connecting rods, used for transmission; each connecting rod is hinged by a rotating shaft to form a connecting rod mechanism, and the connecting rod mechanism is fixed on the bottom base to form the mechanical claw 401;

[0034] The lead screw nut 403 is composed of a nut and a lead screw. Through the combination of the screw pair, when the lead screw rotates, the nut moves linearly on the lead screw, realizing the conversion from rotational motion to linear motion. The mechanical claw 401 is used to grasp the sea buckthorn tree 801. The bottom connecting rod of the connecting rod mechanism of the mechanical claw 401 is fixedly connected with the nut of the lead screw nut 403 by screws. The lead screw of the lead screw nut 403 and the output shaft of the lead screw motor 405 are connected by a coupling. The mechanical claw 401 is installed on the bottom base through Figure 4 On the rectangular sheet metal cover 406 shown in the figure, when the screw motor 405 is working, the rotational motion is converted into linear motion through the screw nut, driving the bottom connecting rod of the mechanical claw 401 as the prime mover, driving the mechanical claw 401 to achieve the action of grasping or releasing; the acceleration sensor 402 is installed on the mechanical claw 401 by screws, and the polarization motor 404 is arranged below the screw motor 405 and installed Figure 4 On the rectangular sheet metal cover 406 shown, when the polarization motor 404 is working, vibration will be generated during rotation due to the eccentricity of the motor shaft. Since the mechanical claw 401 and the polarization motor 404 are both mounted on the rectangular sheet metal cover 406, the vibration can be transmitted from the polarization motor 404 to the mechanical claw 401 via the rectangular sheet metal cover 406, thereby driving the mechanical claw 401 to vibrate.

[0035] Furthermore, a harvesting method of a sea buckthorn harvesting device using a mechanical arm vibration-assisted automatic sea buckthorn harvesting device based on machine vision comprises:

[0036] A method for locating a seabuckthorn tree 801 and a grasping point of a mechanical claw 401 based on machine vision, the steps of which are as follows:

[0037] D100, Preliminary preparation work, using deep learning models such as YOLOv5 for object detection. First, collect a large amount of image data of seabuckthorn trees 801 in the mature stage. Use annotation tools to annotate the tree trunks, branches, and seabuckthorn fruits in the images. The annotation content includes the category of the object and the position of the bounding box. Then, divide the dataset. The annotated data will be used to train the model so that the model can learn the features and position information of each part of the seabuckthorn tree 801. Finally, export the trained and optimized model into a suitable format and then import it into the embedded processor 203 system.

[0038] D200, Reasonably arrange the position of the binocular camera. Install the binocular camera on the frame of the fruit dropping collection device 7. According to the viewing angle of the binocular camera, adjust the installation height and position so that the entire seabuckthorn tree 801 can be included in the field of view when the camera is far away, and the entire tree crown can also be photographed when it is close, and the size of the body of the fruit dropping collection device 7 can be used as a coordinate reference.

[0039] D300, Adjust the parameters of the camera, such as focal length, aperture, shutter speed, and ISO, etc., to ensure the image quality. On a sunny day with sufficient light, the ISO can be appropriately reduced and the shutter speed can be increased to reduce image noise. At the same time, consider the shooting angle and try to avoid backlighting shooting to avoid affecting the feature extraction of the seabuckthorn tree 801.

[0040] D400, The binocular camera takes pictures continuously at a frequency of 50Hz. When the harvesting device is just placed at the starting point between the ridges, according to the viewing angle, the images that can be collected at this time are mainly the ridges and the edges of the seabuckthorn tree 801. As the harvesting device continues to move forward, the entire image of the seabuckthorn tree 801 can be collected. When the harvesting device is in the harvesting position, the binocular camera can collect the images of the entire tree crown and the seabuckthorn fruits on it.

[0041] D500, Preprocess the images of the seabuckthorn tree 801 collected by the binocular camera, including:

[0042] Grayscale processing: Convert the RGB color image of the seabuckthorn tree 801 directly captured by the binocular camera into a grayscale image using the weighted average method to reduce the amount of data and the subsequent calculation complexity.

[0043] Filtering and denoising: There may be noise in the images of the seabuckthorn tree 801, which affects the positioning accuracy of the seabuckthorn tree 801. Filters can be used to remove the noise.

[0044] Image enhancement: Enhance the contrast of the images of the seabuckthorn tree 801 through methods such as histogram equalization to make the contours and details of the seabuckthorn tree 801 clearer.

[0045] D600, the shape of Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit has certain characteristics. The crown of Hippohgae rhamnoides L. tree 801 is generally irregularly circular or oval, while Hippohgae rhamnoides L. fruit is generally a cluster of irregularly circular fruits. Through edge detection algorithms, such as the Canny edge detection algorithm, the edge contours of Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit are extracted. Canny edge detection includes steps such as Gaussian filtering, calculating gradient magnitude and direction, non-maximum suppression, and double-threshold detection. After extracting the edges, through fitting algorithms (such as least squares ellipse fitting) to further describe the shape of Hippohgae rhamnoides L. tree 801, and parameters such as the major axis, minor axis, and center position of Hippohgae rhamnoides L. fruit, so as to distinguish Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit.

[0046] D700, using the pre-trained YOLOv5 deep learning model, it learns various features and patterns of Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit through a large amount of training data, and can automatically extract the areas of Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit in the image and determine their positions. It divides the processed image into multiple grids, and each grid is responsible for predicting whether there is a bounding box of Hippohgae rhamnoides L. tree 801 or Hippohgae rhamnoides L. fruit. Through one forward propagation, the position information of all Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit can be obtained.

[0047] D800, from the above steps, the position information of Hippohgae rhamnoides L. tree 801 and Hippohgae rhamnoides L. fruit is obtained. According to the center point coordinates of the bounding box of Hippohgae rhamnoides L. fruit, combined with the edge contour features of the branch of Hippohgae rhamnoides L. tree 801, the center point coordinates of the bounding box of Hippohgae rhamnoides L. fruit are moved downward along the branch of Hippohgae rhamnoides L. tree 801 by 20 - 30 cm as required to obtain a new coordinate, and this coordinate is set as the grasping point for harvesting this part of Hippohgae rhamnoides L. fruit.

[0048] D900, the embedded processor 203 further processes the position information of Hippohgae rhamnoides L. tree 801, Hippohgae rhamnoides L. fruit, and the grasping point. The reasonably arranged binocular vision system 5 can photograph a part of the harvesting device and measure the size of this part as a benchmark for distance calculation. According to each position information, the distance between the harvesting device and these positions can be calculated. At the same time, a coordinate system is established with the harvesting device as the center, and the angular relationship between Hippohgae rhamnoides L. tree 801 and the harvesting device can also be obtained from the position relationship. Knowing the angle and distance, the precise positioning of Hippohgae rhamnoides L. tree 801 can be achieved.

[0049] D1000, using the obtained relative position relationship, the embedded processor 203 cooperates with the PLC 202 to achieve accurate control of each device and ensure the efficient operation of the harvesting device.

[0050] A method for automatically controlling the vibrating harvesting of Hippohgae rhamnoides L. fruit by a mechanical claw 401, the steps of which are:

[0051] C100. Obtain the spatial coordinates of the grasping points by the above method, perform path planning by the embedded processor 203, and transmit the result to the PLC 202 through serial communication. The PLC 202 controls each joint motor to make the picking manipulator 3 move and deliver the end effector to the grasping point.

[0052] C200. After the end effector reaches the grasping point, the lead screw motor 405 rotates, and the mechanical claw 401 clamps onto the branch.

[0053] C300. The vibration excitation power supply 204 supplies power to the polarization motor 404, and the end effector device 4 starts to vibrate.

[0054] C400. The vibration excitation power supply 204 sweeps the frequency from 10 Hz to 500 Hz as set. An acceleration sensor 402 is installed on the end effector device 4. The acceleration sensor 402 measures the vibration amplitude of the branch. After the frequency sweep is completed, record the frequency at which the branch reaches the maximum vibration amplitude, and control the vibration excitation power supply 204 to continuously output at this frequency, and officially start the sea buckthorn harvesting.

[0055] C500. After harvesting for five minutes, the binocular vision system 5 collects sea buckthorn fruit pictures and compares them with the sea buckthorn fruit pictures at the same position before starting the picking in the above steps. If the sea buckthorn fruit harvesting deficiency rate is less than 90%, increase the voltage of the electrical signal output by the vibration excitation power supply 204. The initial value of the voltage of the electrical signal output by the vibration excitation power supply 204 is 24 V, and it increases by 6 V each time, with a maximum of 48 V, to increase the vibration amplitude of the end effector device 4.

[0056] C600. Loop through step S500 until the sea buckthorn fruit harvesting rate is greater than 90%. Then, the vibration excitation power supply 204 stops supplying power to the polarization motor 404, and the PLC 202 controls the picking manipulator 3 to return along the original path to the starting point and wait for the next cycle.

[0057] An automatic sea buckthorn harvesting device with mechanical arm vibration assistance based on machine vision according to the present invention performs fully automated and efficient sea buckthorn harvesting by using the above device and method, including the following steps:

[0058] S100. Place the harvesting device between the ridges, make the vehicle body as parallel to the field ridges as possible, start the gasoline generator 205, and press the start button. The harvesting device starts to work.

[0059] S200. The crawler moving device 1 moves, and the harvesting device advances at a constant speed. At the same time, the binocular vision system 5 collects images at a frequency of 50 Hz and transmits the data to the embedded processor 203.

[0060] S300. The embedded processor 203 processes the image acquisition data. When sea buckthorn is recognized, the processor uses the image data to calculate the spatial position relationship between the sea buckthorn tree 801 and the harvesting device.

[0061] S400. The processed spatial position relationship is transmitted to PLC202 via serial communication. PLC202 controls the differential rotation of the stepping motors of the crawler moving device 1 to turn the harvesting device. The binocular vision system 5 continuously collects image data, and the processor calculates the spatial position relationship. When the center line of the seabuckthorn tree 801 is aligned with the advancing direction of the harvesting device, the turning stops and the harvesting device moves forward.

[0062] S500. When the proximity sensor 701 of the fruit collection device 7 recognizes the specified distance, the signal is sent back to PLC202, and the crawler moving device 1 stops operating, and the harvesting device reaches the harvesting position.

[0063] S600. After the harvesting device reaches the harvesting position, PLC202 controls the two motors of the collection umbrella device 6 to fully deploy the umbrella cloth 603 to surround the seabuckthorn tree 801.

[0064] S700. After the umbrella cloth 603 is fully deployed, the binocular vision system 5 determines the spatial coordinates of the first grasping point according to the specified process and transmits the spatial coordinates to PLC202.

[0065] S800. PLC202 controls the operation of the picking robotic arm 3. Combining the binocular vision system 5 and the embedded processor 203, it automatically plans the travel path of the picking robotic arm 3, avoids obstacles and successfully sends the end effector 4 to the grasping point.

[0066] S900. PLC202 controls the rotation of the lead screw motor 405 on the end effector 4 that drives the mechanical claw 401 to firmly clamp the grasping point. Then the polarization motor 404 is started, and the mechanical claw 401 vibrates to shake off the seabuckthorn fruits, where the polarization motor 404 is controlled according to the above specific method.

[0067] S1000. The shaken-off seabuckthorn fruits fall into the umbrella cloth 603, gather at the notch of the umbrella cloth 603 at the lower position, and then fall into the fruit box 703 below. After the binocular vision system 5 recognizes that the seabuckthorn fruits at the grasping point have been picked, the mechanical claw 401 releases the branch and the end effector 4 stops operating.

[0068] S1100. After the harvesting of the seabuckthorn fruits at one grasping point is completed, the cycle from S700 to S1000 starts until the binocular vision system 5 cannot recognize the next grasping point on this seabuckthorn tree 801.

[0069] S1200. PLC202 controls the operation of the collection umbrella motor 601 of the collection umbrella device 6 to retract the umbrella cloth 603. Then PLC202 controls the crawler moving device 1 to retreat to the position of S300, and then starts the cycle from S300 to S1200.

[0070] S1300, after the fruit box 703 of the fruit dropping collection device 7 is full, that is, when the pressure sensor 704 reaches the set pressure (the full box weight measured before the harvesting device operates), the end effector 4 stops operating, the umbrella cloth 603 retracts, and the harvesting device retreats to the space between the ridges. When the harvesting device stops operating, the embedded processor 203 sends a signal to the control center to notify that the fruit box 703 is full. After the worker removes the full fruit box 703 and replaces it with an empty fruit box 703, the harvesting device is restarted and continues the harvesting operation.

[0071] The beneficial effects of the present invention are as follows: An automatic seabuckthorn harvesting device with mechanical arm vibration assistance based on machine vision of the present invention realizes efficient and continuous harvesting of large-scale planted seabuckthorn trees 801 in plantations. The crawler moving device 1 is equipped with rubber crawlers, enabling the harvesting device to move freely in the loam environment of the plantation, ensuring the continuity of the harvesting operation. At the same time, using the differential steering function, the device can reach any designated position. The picking robotic arm 3 has six degrees of freedom and can avoid obstacles according to the program, sending the end effector 4 point-to-point to the harvesting position. The end effector 4 vibrates the near-end branches of the seabuckthorn tree 801. Compared with the traditional harvesting method of shaking the tree trunk, it has a better fruit dropping efficiency. At the same time, vibrating the end branches will not damage the tree trunk and roots, will not affect the vitality of the seabuckthorn tree 801, and will not have a negative impact on the fruit yield in the second year. And the end effector 4 is equipped with an acceleration sensor 402 to detect the vibration amplitude of the branches of the seabuckthorn tree 801. Combining with the excitation power supply 204 to output an electrical signal with a suitable frequency, the end effector 4 can have an optimal amplitude, which can significantly increase the fruit dropping rate. Then, combined with the comparison of the binocular vision system 5, the remaining seabuckthorn fruits on the branches can be detected. If there are more remaining seabuckthorn fruits, the amplitude of the electrical signal output by the power supply is further increased, so that the amplitude of the end effector 4 is further increased, shaking off the remaining seabuckthorn fruits and improving the harvesting rate. The collection umbrella device 6 works in coordination, unfolds following the production rhythm when the end effector 4 is working, converges the fallen fruits, and makes them fall into the fruit box 703 of the fruit dropping collection device 7. The fruit box 703 is placed on the pressure sensor 704. Before the weight of the fruit box 703 reaches the set value, the harvesting device will keep working, further improving the harvesting efficiency. After the weight of the fruit box 703 reaches the set value, the harvesting device stops in time, sends a signal to the control center to notify the worker to come and replace the fruit box 703, and can continue to work after restarting after replacement. The devices of the present invention work in coordination, and on the basis of low-intensity work with less labor, can realize automatic, continuous, and efficient harvesting operations for the seabuckthorn trees 801 in the plantation.

[0072] The above are only the preferred embodiments of the present invention, and do not limit the implementation manners and protection scope of the present invention accordingly.

[0073] The present invention also has the following implementation manners on the above basis:

[0074] In a further embodiment of the present invention, Embodiment 1:

[0075] A mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, as Figure 1 shown: It includes a crawler mobile device 1, a control cabinet 2, a picking manipulator 3, an end effector 4, a binocular vision system 5, a collection umbrella device 6, and a fruit-drop collection device 7.

[0076] Among them, as Figure 2 shown, the main components of the crawler mobile device 1 are the right rubber crawler 101, the right stepping motor 102, the frame 103, the left stepping motor 104, the left rubber crawler 105, and the mounting plate 106. The movement direction of the crawler mobile device 1 can be controlled by two stepping motors using the PLC 202, enabling it to move to a specified position. At the same time, the rubber crawlers allow the crawler mobile device 1 to move smoothly in the plantation. The frame 103 is a frame structure welded by channel steel, and the mounting 106 is welded on the frame 103.

[0077] As Figure 2 shown, the control cabinet 2 mainly includes a sheet metal cover 201, a PLC 202, an embedded processor 203, a vibration excitation power supply 204, and a gasoline generator 205. There are openings at the bottom of the sheet metal cover 201, which is fixed to the mounting plate 106 of the crawler mobile device 1 by screws. The PLC 202 and the embedded processor 203 are locked to the side of the sheet metal cover 201 by screws, and the vibration excitation power supply 204 and the gasoline generator 205 are directly fixed to the mounting plate 106 by screws. For each part of the control cabinet 2, the gasoline generator 205 provides power for the entire seabuckthorn harvesting device, having good endurance. It can work for several hours with one refueling, improving the harvesting efficiency. The embedded processor 203 is responsible for processing data from various sensors of the harvesting device, mainly processing and analyzing the images collected by the binocular vision system 5, obtaining the orientation of the harvesting device during operation, and sending the processed data to the PLC 202 to work in coordination with the PLC 202. The PLC 202 is responsible for controlling the actions of all motors on the harvesting device, receiving data from the embedded processor 203, and can directly control the execution of the crawler mobile device 1, the picking manipulator 3, and the end effector 4 to ensure the smooth harvesting rhythm. The vibration excitation power supply 204 is connected to the gasoline generator and directly supplies power to all electrical appliances on the harvesting device. At the same time, it can control the working condition of the end effector 4 by outputting electrical signals with different frequencies and amplitudes.

[0078] As Figure 1As shown in the figure, the picking robotic arm 3 is fixed on the mounting plate 106 of the crawler mobile device 1 by screws. The picking robotic arm 3 is equipped with six motors, providing six degrees of freedom. When harvesting seabuckthorn fruits, it can bend freely in space to avoid seabuckthorn branches, and accurately send the end effector 4 from the starting point to the calculated grasping point. After completing the harvesting task at this grasping point, it can return to the starting point along the original path.

[0079] For the end effector 4, as Figure 4 shown, the main working parts include a mechanical claw 401, an acceleration sensor 402, a lead screw nut 403, a polarization motor 404, and a lead screw motor 405. The bottom of the end effector 4 can be directly connected to the picking robotic arm 3. The grasping surface at the top of the mechanical claw 401 is arc-shaped, which is convenient for grasping branches. The acceleration sensor 402 is locked to the base of the mechanical claw 401 by screws, and can directly detect the vibration amplitude of the seabuckthorn branches during operation, facilitating the embedded processor 203 to detect the working condition of the end effector 4 and timely adjust the output of the excitation power supply 204. The bottom link of the linkage mechanism of the mechanical claw 401 is connected to the nut of the lead screw nut 403, and the lead screw motor 405 is connected to the lead screw of the lead screw nut 403 through a coupling. When the PLC 202 controls the rotation of the lead screw motor 403, due to different rotation directions, the mechanical claw 401 can be controlled to perform the grasping or releasing action. The polarization motor 404 is installed at the bottom of the mechanical claw 401. When the polarization motor 404 works, because its motor shaft is eccentric, it will generate regular vibrations to vibrate the seabuckthorn branches, and this regularity is directly controlled by the excitation power supply 204 and can be controlled by a program.

[0080] The binocular vision system 5 mainly consists of two binocular cameras with image acquisition cards, the first binocular camera 501 and the second binocular camera 502, as Figure 8 and Figure 9 shown. Because the seabuckthorn tree has many and dense branches, by reasonably arranging the positions of the two binocular cameras, the field of view of the binocular vision system 5 is expanded. And when there are many branch obstructions, the images of the two binocular cameras can complement each other, reducing blind spots. Since position information needs to be recognized to calculate distances, arranging two binocular cameras can also expand the effective range of depth measurement and improve the accuracy of depth measurement.

[0081] The collection umbrella device 6 mainly consists of a collection umbrella motor 601, an umbrella frame 602, and an umbrella cloth 603, as Figure 5As shown in the figure, two collecting umbrella motors 601 are connected to the umbrella frames 602 at both ends through couplings. When the PLC 202 controls the rotation of the two collecting umbrella motors 601, the collecting umbrella motors 601 can be used to control the unfolding or retraction of the umbrella cloth 603, which is convenient for automatic control. The umbrella frames 602 are inserted into the umbrella sleeves of the umbrella cloth 603 and fixed on both sides with ropes. The umbrella frames 602 are placed at the designated positions of the fruit drop collection device 7 and are equipped with linear bearings for easy rotation. They can be installed by fixing with snap rings on both sides.

[0082] For the fruit drop collection device 7, as Figure 6 shown, it mainly consists of a proximity sensor 701, a collecting vehicle 702, a fruit box 703, a pressure sensor 704, and a baffle 705. The proximity sensor 701 and the pressure sensor 704 are both fixedly connected to the collecting vehicle 702 by screws. The collecting vehicle 702 is connected to the crawler moving device 1 by bolts. The fruit box 703 is placed in the middle of the collecting vehicle 702. Because it works in a loess environment and there is a concern that the shaking of the harvesting device may cause the fruit box 703 to come out, there are baffles 705 on both sides to limit the fruit box 703.

[0083] In a further embodiment of the present invention, Embodiment 2:

[0084] A visual positioning method for a robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision according to Embodiment 1 includes:

[0085] As Figure 11 shown in the positioning work process, for the pre-preparation work, the YOLOv5 deep learning model is required for object detection. First, a large number of image data of seabuckthorn trees in the mature period from September to October are collected. At this time, the leaves of the seabuckthorn trees have basically fallen off, that is, there are only seabuckthorn trees and seabuckthorn fruits in the images, as Figure 12 shown in the photos in Figure 13 . The annotation tool is used to annotate the seabuckthorn trees and seabuckthorn fruits in the images. The annotation content includes the category of the object and the position of the bounding box. Then, the data set is divided. The annotated data will be used to train the model to let the model learn the features and position information of each part of the seabuckthorn tree. Finally, the trained and optimized model is exported in a suitable format and then imported into the embedded processor system 203.

[0086] As Figure 8 and Figure 9As shown in the figure, since the viewing angle of a common binocular camera is generally 90°, in order to capture the images of the seabuckthorn trees required, it is necessary to reasonably arrange the positions of the binocular cameras. First, it is selected to install the first binocular camera 501 and the second binocular camera 502 on the collection vehicle 702 of the fruit drop collection device 7 because there is less likely to be occlusion at this position. Then, according to the 90° viewing angle of the binocular camera, adjust the installation height and position so that the binocular camera can capture the entire seabuckthorn tree in the field of view when it is far away and can also cover the entire tree crown when it is close, as Figure 10 and Figure 11 shown, and the size of the body of the fruit drop collection device 7 can be used as a coordinate reference.

[0087] Reasonably adjust the parameters of the first binocular camera 501 and the second binocular camera 502, such as focal length, aperture, shutter speed, and ISO, etc., to ensure the quality of the captured seabuckthorn tree images. On a sunny day with sufficient light, the ISO can be appropriately reduced and the shutter speed increased to reduce image noise so as not to affect the subsequent feature extraction of the seabuckthorn tree.

[0088] The first binocular camera 501 and the second binocular camera 502 continuously take pictures at a frequency of 50 Hz. As Figure 6 shown, when the harvesting device is at the starting position 802 of the harvesting device, according to the viewing angle, the images that can be captured at this time are mainly the ridges and the edges of the seabuckthorn trees, and there is no complete seabuckthorn tree image, so the YOLOv5 deep learning model cannot recognize the seabuckthorn tree. As the harvesting device continues to move forward, the entire image of the seabuckthorn tree can be captured, as Figure 12 shown. When the harvesting device is in the harvesting position, the binocular camera can capture the images of the entire tree crown and the seabuckthorn fruits on it, as Figure 13 shown.

[0089] Preprocess the seabuckthorn tree images captured by the first binocular camera 501 and the second binocular camera 502, including: Grayscale processing: Convert the RGB color image of the seabuckthorn tree directly captured by the binocular camera into a grayscale image using the weighted average method to reduce the amount of data and the subsequent computational complexity. Filtering and denoising: There may be noise in the seabuckthorn tree image, which affects the positioning accuracy of the seabuckthorn tree. Filters can be used to remove the noise. Image enhancement: Enhance the contrast of the seabuckthorn tree image through methods such as histogram equalization to make the contours and details of the seabuckthorn tree and the seabuckthorn fruits clearer.

[0090] The shapes of seabuckthorn trees and seabuckthorn fruits have certain characteristics. The crown of a seabuckthorn tree is generally irregularly circular or elliptical, while seabuckthorn fruits are generally clusters of irregularly circular fruits. The edge contours of seabuckthorn trees and seabuckthorn fruits are extracted through edge detection algorithms, such as the Canny edge detection algorithm. Canny edge detection includes steps such as Gaussian filtering, calculating the gradient magnitude and direction, non-maximum suppression, and double-threshold detection. After extracting the edges, the shape of the seabuckthorn tree and parameters such as the major axis, minor axis, and central position of the seabuckthorn fruit are further described through fitting algorithms (such as least squares ellipse fitting), so as to distinguish seabuckthorn trees and seabuckthorn fruits.

[0091] Using the previously trained YOLOv5 deep learning model, which learns various features and patterns of seabuckthorn trees and seabuckthorn fruits through a large amount of training data, it can automatically extract the regions of seabuckthorn trees and seabuckthorn fruits in the image and determine their positions. It divides the processed image into multiple grids, and each grid is responsible for predicting whether there are bounding boxes of seabuckthorn trees or seabuckthorn fruits, such as Figure 12 and Figure 13 shown. The rectangular boxes in the figure are the bounding boxes of seabuckthorn trees and seabuckthorn fruits recognized by the YOLOv5 deep learning model, and the position information of all seabuckthorn trees and seabuckthorn fruits can be obtained through one forward propagation.

[0092] From the above steps, the position information of seabuckthorn trees and seabuckthorn fruits is obtained. According to the center point coordinates of the bounding box of the seabuckthorn fruit and combined with the edge contour features of the seabuckthorn branch where it is located, the center point coordinates of the bounding box of the seabuckthorn fruit are moved downward along the seabuckthorn branch by 20 - 30 cm as required to obtain a new coordinate, and this coordinate is set as the grasping point for harvesting this part of the seabuckthorn fruit, such as Figure 13 shown. The positions marked by the red dots are the grasping points corresponding to the two clusters of seabuckthorn fruits on the branch.

[0093] The embedded processor 203 further processes the position information of seabuckthorn trees, seabuckthorn fruits, and grasping points. The reasonably arranged binocular vision system 5 can photograph a part of the harvesting device and measure the size of this part as a benchmark for distance calculation. According to the respective position information, the distance between the harvesting device and these positions can be calculated. At the same time, a coordinate system is established with the harvesting device as the center, and the angular relationship between the seabuckthorn tree and the harvesting device can also be obtained from the position relationship. Knowing the angle and distance, precise positioning of the seabuckthorn tree can be achieved. Using the obtained relative position relationship, the embedded processor 203 collaborates with the PLC 202 to accurately control each device and ensure the efficient operation of the harvesting device.

[0094] In a further embodiment of the present invention, Embodiment 3:

[0095] A harvesting method based on the robotic arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision described in Embodiment 1 and the vision positioning method described in Embodiment 2, comprising:

[0096] Combined with Figure 10 the overall workflow of, Embodiment 3 is described. When the harvesting device starts to work, as Figure 7 shown, the harvesting device is placed at the starting position 802 of the harvesting device in the plantation, ensuring that the vehicle body is roughly parallel to the ridge edge of the seabuckthorn tree. The worker presses the start button on the control box 2, and the harvesting device is powered on. The crawler moving device 1 starts to act, and the two side stepping motors rotate at the same speed, driving the harvesting device forward. At the same time, as Figure 6 shown, the second binocular camera 502 arranged and set as required, and the binocular camera 503 start to work for image acquisition. During the forward movement of the harvesting device, the seabuckthorn tree 801 will gradually enter the field of view of the binocular vision system 5. As Figure 12 shown, when the entire seabuckthorn tree 801 is captured by the second binocular camera 502 and the binocular camera 503, based on the method in Embodiment 2, the distance between the seabuckthorn tree 801 and the harvesting device can be obtained. At the same time, with the harvesting device as the center, a coordinate system is established in the forward direction and the vertical direction of the harvesting device to obtain the angular relationship between the seabuckthorn tree and the harvesting device. Knowing the angle and distance, precise positioning of the seabuckthorn tree can be achieved. At this time, the harvesting device continues to move forward. When the angle between the line connecting the harvesting device and the target seabuckthorn tree 801 and the forward direction of the harvesting device is about 45°, the crawler moving device 1 stops. Assuming that the seabuckthorn tree 801 is at 45° clockwise in the forward direction of the harvesting device, at this time, the right stepping motor 102 of the crawler moving device 1 stops acting, and the left stepping motor 104 starts to rotate forward. The left rubber crawler 105 rotates clockwise around the right rubber crawler 101, and the entire harvesting device rotates clockwise. The seabuckthorn tree 801 rotates counterclockwise relative to the harvesting device. When the seabuckthorn tree 801 is facing the harvesting device directly, the right stepping motor 102 also starts to rotate forward at the same speed, and the harvesting device stops the rotation action and starts to move straight. The harvesting device moves forward. When the distance between the fruit collection device 7 and the seabuckthorn tree 801 reaches the set value (the appropriate distance obtained through multiple measurements to make the seabuckthorn tree exactly fall in the middle of the collection umbrella device 6), the proximity sensor 701 sends a signal to the embedded sensor, indicating that the harvesting device has reached Figure 7 the harvesting position 803 of the harvesting device shown in. After reaching the harvesting position 803, the crawler moving device 1 stops, and the collection umbrella motor 601 of the collection umbrella device 6 receives the instruction from the PLC 202 and starts to rotate, fully unfolding the umbrella cloth 603. At this time, the photo that the binocular vision system 5 can capture should be the entire canopy of the seabuckthorn tree, and the seabuckthorn fruits can be seen relatively clearly, as Figure 13As shown in the figure, a part of the tree crown is intercepted, and the spatial coordinates of the grasping point are obtained through the positioning method described in Embodiment 2, and then the spatial position relationship between the grasping point and the harvesting device is obtained. Further, the embedded processor 203 performs spatial path planning based on this spatial position relationship and the images collected by the binocular vision system 5 at the location where it is located, and converts this plan into the motion conditions of the joint motors of the picking robotic arm 3. Then, the instructions are transmitted to the PLC 202 through serial communication, and the PLC 202 directly controls the actions of each motor to send the end effector 4 from the starting point to the grasping point. After the end effector 4 reaches the grasping point, the joint motors of the picking robotic arm 3 are locked, and the PLC 202 controls the lead screw motor 405 to rotate, so that the mechanical claw 401 tightly grasps the seabuckthorn branch. Then, the vibration excitation power supply 204 starts to supply power to the polarization motor 404, and the vibration excitation power supply 204 performs frequency sweeping from 10 Hz to 500 Hz. At the same time, the acceleration sensor 402 continuously detects the vibration amplitude of the seabuckthorn branch. After one frequency sweep is completed, the embedded processor 203 automatically records the electrical signal frequency corresponding to the maximum vibration amplitude, and then controls the vibration excitation power supply 204 to continuously output electrical signals at this frequency, and the end effector 4 starts to work officially. Because the shapes of seabuckthorn branches are strange and each seabuckthorn branch is different, the vibration effects produced by the same frequency acting on them will be different. Therefore, a frequency sweeping technical method is added here, so that the end effector 4 can automatically work at the frequency most suitable for the current seabuckthorn branch, avoiding damage to the seabuckthorn branch caused by inappropriate frequencies and improving the harvesting efficiency at the same time. After working officially for five minutes, the end effector 4 pauses working, and the embedded processor 203 compares the current seabuckthorn fruit image with the seabuckthorn fruit image five minutes ago, calculates the harvesting rate. If the harvesting rate is less than 90%, the amplitude of the electrical signal output by the vibration excitation power supply 204 is increased, and the end effector 4 starts to work again, shaking the seabuckthorn branch with a larger vibration amplitude. After five minutes, the comparison and calculation are performed again, and the cycle is carried out until the harvesting rate is greater than 90%. A harvesting rate greater than 90% means that the seabuckthorn fruits at this grasping point have been harvested. The fruits shaken off by the end effector 4 fall on the unfolded umbrella cloth 603. The umbrella cloth 603 has openings at a lower position and is aligned with the fruit box 703 of the falling fruit collection device 7 below, completing the collection of the seabuckthorn fruits at this grasping point. Then, the PLC 202 controls the lead screw motor 405 to act and controls the mechanical claw 401 to loosen. Then, according to the spatial coordinates of the second grasping point, the spatial relative position relationship between the two grasping points is calculated. Then, the harvesting method for the first grasping point is the same as above, and will not be elaborated here. The cycle is carried out in sequence until all the grasping points of this seabuckthorn tree have been executed through the above work process, and the harvesting work of this seabuckthorn tree is considered completed.After the harvesting of the first seabuckthorn tree is completed, the collecting umbrella motor 602 operates to retract the umbrella cloth 603. Then, the crawler moving device 1 starts to operate, and the two stepping motors start to reverse. The harvesting device retreats to the turning position, resumes the original forward direction, and prepares to start the harvesting of the second seabuckthorn tree. The harvesting method is the same as that of the first tree. The harvesting device harvests the seabuckthorn trees one by one in sequence in the plantation according to the above method. When the fruit box 703 of the fruit dropping collection device 7 is full, that is, when the pressure sensor 704 reaches the set pressure (the full box weight measured before the harvesting device operates), the end effector 4 stops operating, the collecting umbrella device 6 retracts, and the harvesting device retreats between the ridges. That is, as shown in. Figure 12 At the harvesting device pause position 804 shown, the harvesting device stops operating. The embedded processor 203 sends a signal to the control center to notify that the fruit box 703 is full. After the worker removes the full fruit box 703 and replaces it with an empty fruit box 703, the harvesting device is restarted, and the device continues to perform the harvesting operation.

[0097] The left driving and walking mechanism includes: a left stepping motor 104, a left walking wheel frame, a left rubber crawler 105, a left driving wheel, and a plurality of left driven wheels. One left driving wheel and a plurality of left driven wheels are rotatably installed on the left walking wheel frame. The left walking wheel frame and the vehicle frame 103 are rotatably connected through a connecting rotating shaft. The plurality of left driven wheels are arranged side by side. The left driving wheel is arranged above the plurality of left driven wheels. The left rubber crawler 105 is wound around the left driving wheel and the plurality of left driven wheels and is in a tensioned state. The left stepping motor 104 is installed on the left walking wheel frame. The left stepping motor 104 drives the left driving wheel to rotate, thereby realizing the walking function of the left rubber crawler 105. The left driving and walking mechanism further includes: a left tensioning wheel and a left tensioning motor. The left tensioning wheel is arranged at the front end or the rear end of the plurality of left driven wheels. The left tensioning motor is installed on the left walking wheel frame. The left tensioning motor is used to adjust the position of the left tensioning wheel to make the left rubber crawler 105 in a tensioned state.

[0098] The right-side driving and traveling mechanism includes: a right-side stepping motor 102, a right-side traveling wheel frame, a right-side rubber crawler 101, a right-side driving wheel, and right-side driven wheels. One right-side driving wheel and multiple right-side driven wheels are rotatably mounted on the right-side traveling wheel frame. The right-side traveling wheel frame and the vehicle frame 103 are rotatably connected through a connecting rotating shaft. The multiple right-side driven wheels are arranged side by side. The right-side driving wheel is disposed above the multiple right-side driven wheels. The right-side rubber crawler 101 is wound around the right-side driving wheel and the multiple right-side driven wheels and is in a tensioned state. The right-side stepping motor 102 is mounted on the right-side traveling wheel frame. The right-side stepping motor 102 drives the right-side driving wheel to rotate, thereby realizing the traveling function of the right-side rubber crawler 101. The right-side driving and traveling mechanism further includes: a right-side tensioning wheel and a right-side tensioning motor. The right-side tensioning wheel is disposed at the front end or the rear end of the multiple right-side driven wheels. The right-side tensioning motor is mounted on the right-side traveling wheel frame. The right-side tensioning motor is used to adjust the position of the right-side tensioning wheel to make the right-side rubber crawler 101 in a tensioned state.

[0099] The above are only the preferred embodiments of the present invention, and thus do not limit the implementation manners and protection scope of the present invention. For those skilled in the art, it should be realized that all the equivalent replacements and obvious changes made by using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision, characterized in that: include: A crawler moving device (1), a control cabinet (2), a picking mechanical arm (3), an end effector (4), a binocular vision system (5), a collecting umbrella device (6) and a fallen fruit collecting device (7), wherein the control cabinet (2) is mounted on the crawler moving device (1), the end effector (4) is mounted on the crawler moving device (1) via the picking mechanical arm (3), the picking mechanical arm (3) is arranged in front of the control cabinet (2), the fallen fruit collecting device (7) is connected to the crawler moving device (1), the binocular vision system (5) and the collecting umbrella device (6) are both mounted on the fallen fruit collecting device (7), and the collecting umbrella device (6) is arranged in front of the binocular vision system (5); The crawler moving device (1) is used to realize the forward movement, backward movement and turning of the sea buckthorn harvesting device, the binocular vision system (5) is used to detect the position of the sea buckthorn tree (801) and the sea buckthorn fruit, the end effector (4) is used to grasp the sea buckthorn tree (801) and vibrate the sea buckthorn tree (801) to cause the sea buckthorn fruit to fall, the collecting umbrella device (6) is used to receive the fallen sea buckthorn fruit and transfer it to the fallen fruit collecting device (7), the picking mechanical arm (3) is used to adjust the working position and angle of the end effector (4), and the control cabinet (2) is used to realize the automatic control of the sea buckthorn harvesting device.

2. The machine vision-based mechanical arm vibration-assisted automatic sea buckthorn harvesting device according to claim 1, characterized in that: The crawler moving device (1) comprises: a vehicle frame (103), a left-side driving walking mechanism, a right-side driving walking mechanism and a mounting plate (106); the left-side driving walking mechanism and the right-side driving walking mechanism are respectively mounted on the left and right sides of the vehicle frame (103) and driven independently; and the mounting plate (106) is mounted on the top of the first vehicle frame (103); The left-side driving walking mechanism and the right-side driving walking mechanism are both crawler-driven walking mechanisms; The left-side driving walking mechanism comprises: a left-side stepping motor (104) and a left-side rubber crawler (105), wherein the left-side stepping motor (104) is used to drive the left-side rubber crawler (105) to walk; the right-side driving walking mechanism comprises: a right-side rubber crawler (101) and a right-side stepping motor (102), wherein the right-side stepping motor (102) is used to drive the right-side rubber crawler (101) to walk.

3. The machine vision-based mechanical arm vibration-assisted automatic seabuckthorn harvesting device according to claim 2, characterized in that: The control cabinet (2) comprises: a sheet metal cover (201), a PLC (202), an embedded processor (203), an excitation power supply (204) and a gasoline generator (205); the sheet metal cover (201) is mounted on the upper surface of a mounting plate (106); the PLC (202), the embedded processor (203), the excitation power supply (204) and the gasoline generator (205) are all arranged in the sheet metal cover (201).

4. The machine vision-based mechanical arm vibration-assisted automatic sea buckthorn harvesting device according to claim 3, characterized in that: The end effector (4) comprises: a mechanical claw (401), an acceleration sensor (402), a lead screw nut (403), a polarization motor (404), a lead screw motor (405) and a rectangular sheet metal cover (406); the mechanical claw (401) is used to grasp the sea buckthorn tree (801); the lead screw motor (405) is used to drive the mechanical claw (401) to achieve a grasping or loosening action; the acceleration sensor (402) is mounted on the mechanical claw (401); the polarization motor (404) and the mechanical claw (401) are both mounted on the rectangular sheet metal cover (406); the polarization motor (404) is arranged below the lead screw motor (405); and the polarization motor (404) is used to drive the mechanical claw (401) to vibrate; The mechanical claw (401) comprises: a top connecting rod (4011), a limiting connecting rod (4012), a transmission connecting rod (4013) and a driving connecting rod (4014); the lead screw nut (403) comprises: a lead screw and a nut, the nut being mounted on the lead screw, the mechanical claw (401) being symmetrically arranged, the rear ends of the two driving connecting rods (4014) being rotationally connected to the outer periphery of the nut, the rear end of each transmission connecting rod (4013) being rotationally connected to the front end of a driving connecting rod (4014), the front end of each transmission connecting rod (4013) being rotationally connected to the rear end of a top connecting rod (4011), the rear end of each limiting connecting rod (4012) being rotationally connected to a rectangular sheet metal cover (406), and the front end of each limiting connecting rod (4012) being rotationally connected to the middle of a top connecting rod (4011).

5. The machine vision-based mechanical arm vibration-assisted automatic sea buckthorn harvesting device according to claim 4, characterized in that: The fallen fruit collection device (7) comprises: a proximity sensor (701), a collection vehicle (702), a fruit frame (703), a pressure sensor (704) and a baffle (705). The rear end of the collection vehicle (702) is connected to the front end of the vehicle frame (103). The collection vehicle (702) is a frame structure with an open top. The interior of the collection vehicle (702) is used to place the fruit frame (703). Baffles (705) are installed on both the left and right sides of the collection vehicle (702). Both baffles (705) are used to limit the position of the fruit frame (703). The pressure sensor (704) is installed in the collection vehicle (702). The pressure sensor (704) is used to detect the quality of the fruit frame (703). The proximity sensor (701) is installed on the front side of the collection vehicle (702). The proximity sensor (701) is used to detect the distance between the sea buckthorn tree (801) and the collection vehicle (702).

6. The machine vision-based mechanical arm vibration-assisted automatic sea buckthorn harvesting device according to claim 5, characterized in that: The collecting umbrella device (6) comprises: an umbrella collecting motor (601), an umbrella frame (602) and an umbrella cloth (603); the umbrella frame (602) comprises: a U-shaped base and a plurality of umbrella rods; the U-shaped base is mounted at the middle of the front side of the top of the collecting vehicle (702); the plurality of umbrella rods are arranged along the upper surface of the U-shaped base; the plurality of umbrella rods are rotatably mounted on the U-shaped base; two umbrella collecting motors (601) are respectively mounted at two ends of the U-shaped base; the two umbrella collecting motors (601) are respectively used to drive the umbrella rods at two ends of the U-shaped base to rotate; the plurality of umbrella rods are inserted into the umbrella cloth (603); the umbrella cloth (603) is enclosed around the sea buckthorn tree (801) and the umbrella cloth (603) is gathered by driving the umbrella rods at two ends of the U-shaped base to rotate through the umbrella collecting motor (601).

7. The machine vision-based mechanical arm vibration-assisted automatic sea buckthorn harvesting device according to claim 6, characterized in that: The binocular vision system (5) comprises: a first binocular camera (501) and a second binocular camera (502), wherein the first binocular camera (501) and the second binocular camera (502) are respectively installed at the left and right ends of the rear side of the top of the collection vehicle (702); the first binocular camera (501) and the second binocular camera (502) are both higher than the upper surface of the umbrella cloth (603), and the viewing angles of the first binocular camera (501) and the second binocular camera (502) are adjustable.

8. The machine vision-based mechanical arm vibration-assisted automatic seabuckthorn harvesting device according to claim 7, characterized in that: The picking mechanical arm (3) comprises: a bottom support rod (31), a middle connecting rod (32), a top connecting rod (33) and a top connecting flange (34); the bottom support rod (31) is perpendicular to the upper surface of the mounting plate (106); the lower end of the bottom support rod (31) and the mounting plate (106) are rotatably connected via a rotating seat; the lower end of the middle connecting rod (32) and the upper end of the bottom support rod (31) are rotatably connected via a rotating shaft; a slide rail is provided on the side of the top connecting rod (33); the rotating shaft seat is rotatably mounted on the top end of the middle connecting rod (32); the rotating shaft seat and the slide rail of the top connecting rod (33) are slidably connected; the top connecting flange (34) is rotatably mounted on the top end of the top connecting rod (33) via a rotating shaft; and the end actuator (4) is rotatably mounted on the top connecting flange (34) via a rotating shaft; The picking robot arm (3) also includes: a first drive motor (35), a second drive motor (36), a third drive motor (37), a fourth drive motor, a fifth drive motor (38) and a sixth drive motor (39), wherein the first drive motor (35) is used to drive the rotating seat to realize the rotation of the bottom support rod (31) around its own axis, the second drive motor (36) is used to drive the lower end of the middle connecting rod (32) to rotate around the rotating shaft, the third drive motor (37) is used to drive the top connecting rod (33) to rotate around the rotating shaft seat, the fourth drive motor is used to drive the rotating shaft seat to slide along the slide rail of the top connecting rod (33), the fifth drive motor (38) is used to drive the top connecting flange (34) to pitch and rotate around the top of the top connecting rod (33), and the sixth drive motor (39) is used to drive the end actuator (4) to rotate around its own axis on the top connecting flange (34).

9. A method for locating the gripping point of a seabuckthorn tree and a mechanical claw based on machine vision, applicable to the mechanical arm vibration-assisted automatic seabuckthorn harvesting device based on machine vision as described in claim 8, the method for locating the gripping point comprising the following steps: D100: Preliminary preparation, using YOLOv5 and other deep learning models for target detection, first, collect a large amount of image data of mature sea buckthorn trees (801), use annotation tools to annotate the trunks, branches and sea buckthorn fruits in the images, and the annotation content includes the category of the object and the position of the bounding box; then, divide the data set, and the annotated data will be used to train the model to let the model learn the characteristics and position information of each part of the sea buckthorn tree (801), and finally export the trained and optimized model into a suitable format, and then import it into the embedded processor (203) system; D200: The binocular camera is arranged in a reasonable position, and is installed on the frame of the fallen fruit collection device (7). The height and position of the installation are adjusted according to the viewing angle of the binocular camera, so that the binocular camera can capture the entire sea buckthorn tree (801) in the field of view when it is far away, and can also capture the entire tree crown when it is close, and the size of the body of the fallen fruit collection device (7) can be used as a coordinate reference; D300: Adjust the camera parameters, such as focal length, aperture, shutter speed, and ISO, to ensure image quality. In the daytime with sufficient light, the ISO can be appropriately lowered and the shutter speed increased to reduce image noise. At the same time, the shooting angle should be considered, and backlight shooting should be avoided as much as possible to avoid affecting the feature extraction of the sea buckthorn tree (801). D400: The binocular camera continuously takes pictures at a frequency of 50 Hz. When the harvesting device is just placed at the starting point between ridges, according to the viewing angle, the images that can be collected at this time are mainly the edges of the ridges and the sea buckthorn trees (801). When the harvesting device continues to move forward, it can collect images of the entire sea buckthorn tree (801). When the harvesting device is in the harvesting position, the binocular camera can collect images of the entire tree crown and the sea buckthorn fruits on it; D500: Preprocess the seabuckthorn tree (801) image captured by the binocular camera, including: Grayscale processing: The RGB color image of the sea buckthorn tree (801) directly captured by the binocular camera is converted into a grayscale image using the weighted average method to reduce the amount of data and the complexity of subsequent calculations; Filtering and denoising: There may be noise in the sea buckthorn tree (801) image, which affects the positioning accuracy of the sea buckthorn tree (801). A filter can be used to remove the noise; Image enhancement: enhancing the contrast of the sea buckthorn tree (801) image by using methods such as histogram equalization, so that the outline and details of the sea buckthorn tree (801) are clearer; D600: The shapes of the seabuckthorn tree (801) and the seabuckthorn fruit have certain characteristics. The crown of the seabuckthorn tree (801) is generally irregularly round or elliptical, and the seabuckthorn fruit is generally irregularly round fruit clustered together. The edge contours of the seabuckthorn tree (801) and the seabuckthorn fruit are extracted by edge detection algorithms, such as the Canny edge detection algorithm; the Canny edge detection includes Gaussian filtering, calculating gradient amplitude and direction, non-maximum suppression and double threshold detection steps; After the edge is extracted, a fitting algorithm (such as least squares ellipse fitting) is used to further describe the shape of the seabuckthorn tree (801) and the parameters of the seabuckthorn fruit, such as the major axis, minor axis, and center position, so as to distinguish the seabuckthorn tree (801) from the seabuckthorn fruit; D700: Using the previously trained YOLOv5 deep learning model, it learns various features and patterns of the sea buckthorn tree (801) and the sea buckthorn fruit through a large amount of training data, and can automatically extract the sea buckthorn tree (801) and the sea buckthorn fruit area in the image and determine their locations; It divides the processed image into multiple grids, each grid is responsible for predicting whether there is a bounding box of a sea buckthorn tree (801) or sea buckthorn fruit, and the location information of all sea buckthorn trees (801) and sea buckthorn fruits can be obtained through one forward propagation; D800: The position information of the sea buckthorn tree (801) and the sea buckthorn fruit is obtained through the above steps. According to the coordinates of the center point of the sea buckthorn fruit boundary box and the edge contour features of the branch of the sea buckthorn tree (801), the coordinates of the center point of the sea buckthorn fruit boundary box are moved downward by 20-30 cm along the branch of the sea buckthorn tree (801) as required to obtain a new coordinate, which is set as the grabbing point for harvesting the sea buckthorn fruit; D900: The embedded processor (203) further processes the position information of the sea buckthorn tree (801), the sea buckthorn fruit, and the grasping point. The properly arranged binocular vision system (5) can capture a part of the harvesting device and measure the size of this part as a basis for distance calculation. According to each position information, the distance between the harvesting device and these positions can be calculated. At the same time, a coordinate system is established with the harvesting device as the center. The angle relationship between the sea buckthorn tree (801) and the harvesting device can also be obtained from the position relationship. Knowing the angle and distance, the sea buckthorn tree (801) can be accurately positioned; D1000: By utilizing the obtained relative position relationship, the embedded processor (203) cooperates with the PLC (202) to achieve accurate control of each device and ensure efficient operation of the harvesting device.

10. A seabuckthorn harvesting method for automatically controlling the vibration of a mechanical claw, applicable to the method for locating the seabuckthorn tree and the gripping point of the mechanical claw based on machine vision as described in claim 9, the seabuckthorn harvesting method comprising the following steps: C100: The spatial coordinates of the grasping point are obtained by the above method, and the embedded processor (203) performs path planning, and transmits the result to the PLC (202) through serial communication. The PLC (202) controls the motors of each joint to move the picking robot arm (3) and deliver the end effector to the grasping point; C200: After the end effector reaches the grasping point, the lead screw motor (405) rotates and the mechanical claw (401) clamps on the branch; C300: The excitation power supply (204) supplies power to the polarization motor (404), and the end effector (4) starts to vibrate; C400: The excitation power supply (204) sweeps the frequency from 10 Hz to 500 Hz according to the setting, and an acceleration sensor (402) is installed on the end effector (4). The acceleration sensor (402) measures the vibration amplitude of the tree branch. After the frequency sweep is completed, the frequency at which the tree branch reaches the maximum vibration amplitude is recorded, and the excitation power supply (204) is controlled to continuously output at the frequency, and the sea buckthorn harvesting officially begins; C500: Five minutes after harvesting, the binocular vision system (5) collects a picture of the sea buckthorn fruit and compares it with the picture of the sea buckthorn fruit in the same position before picking in step C400. If the sea buckthorn fruit harvesting rate is less than 90%, the voltage of the electric signal output by the excitation power supply (204) is increased by 6V each time, so that the vibration amplitude of the end effector (4) is increased and step C400 is repeated; C600: loop step C500 until the sea buckthorn fruit harvesting rate is greater than 90%, then the excitation power supply (204) stops supplying power to the polarization motor (404), and the PLC (202) controls the picking robot arm (3) to return along the original route, return to the starting point, and wait for the next cycle; when the fruit frame (703) of the fallen fruit collecting device (7) is full, that is, when the pressure sensor (704) reaches the set pressure, the end effector (4) stops moving, the umbrella cloth (603) is retracted, and the harvesting device is returned to the ridge; the harvesting device stops moving, and the embedded processor (203) sends a signal to the control center to notify that the fruit frame (703) is full, and the worker takes down the full fruit frame (703) and replaces it with an empty fruit frame (703), restarts the harvesting device, and the harvesting device continues to perform the harvesting operation.

Images