A triangular crawler self-propelled palm fruit harvesting robot based on visual perception and fusion positioning

Abstract

The application discloses a kind of based on visual detection and fusion positioning mapping triangular track self-propelled palm fruit harvesting robot, comprising: walking system, fruit collecting system, picking system and control system.Robot is realized synchronous positioning and mapping by fusing visual information and inertial measurement data, and is fused with satellite navigation positioning information, improve the positioning accuracy and operation stability of robot in complex orchard environment;Walking system is triangular track chassis and is provided with supporting device, ensure the passability and operation stability of sloping ground and uneven ground;Picking system includes rotating device, multistage telescopic lifting mechanical arm and end harvesting device;Fruit collecting system includes collection pipeline and fruit collecting box.The application is combined by lifting type picking structure and visual detection, reduces the dependence on artificial identification in picking process, ensures the accessibility and picking accuracy of high tree crown oil palm fruit, and improves the working continuity of oil palm fruit picking operation.

Description

Technical Field

[0001] This invention relates to the field of agricultural machinery technology, specifically to a triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion positioning. Background Technology

[0002] As an important tropical economic crop, oil palm fruit harvesting is a crucial stage in oil palm production, characterized by high labor intensity, high frequency, and stringent time requirements. Oil palm cultivation areas are extensive in southern my country and Southeast Asia. Oil palm trees typically grow to a height of 5–15 meters, with fruits mainly distributed in the upper and middle parts of the canopy. Harvesting space is significantly limited by tree height and canopy structure. Currently, oil palm fruit harvesting is still primarily manual, usually relying on manual climbing or simple tools. This method is not only labor-intensive and inefficient, but also poses significant safety hazards due to working at heights and the falling fruit, making it difficult to meet the demands for high efficiency and safety under large-scale planting conditions. With rising labor costs, traditional manual harvesting methods have gradually become a major factor restricting the development of the oil palm industry.

[0003] In recent years, with the development of smart agriculture technology, machine vision and autonomous navigation technologies have been gradually applied to orchard operation equipment. However, their application in the field of oil palm fruit harvesting is still in the exploratory stage. Although some existing orchard operation equipment has introduced visual detection technology for target recognition, most of them only achieve simple image acquisition or two-dimensional target judgment, lacking the ability to stably acquire spatial location information of fruits. Recognition stability remains insufficient in oil palm orchard environments with complex canopy obstruction and significant light variations. Furthermore, existing equipment relies heavily on preset routes or manual remote control for movement during operation, lacking autonomous positioning and mapping capabilities based on environmental perception. This makes it difficult to adapt to the complex spatial environment and frequently changing operation paths in oil palm planting areas, thus limiting operational efficiency and safety.

[0004] To address the aforementioned issues, existing technologies include various oil palm fruit harvesting machines or auxiliary harvesting devices, such as fixed-height operating platforms, simple robotic arms with end effectors, or handheld harvesting tools, to reduce manual labor intensity. However, these devices often suffer from poor structural adaptability, with limited adjustable operating heights, making them unsuitable for orchard environments with significant differences in oil palm tree height and varied terrain. Furthermore, some equipment lacks stable support structures, resulting in insufficient stability when operating on slopes or uneven ground, affecting harvesting safety and operational continuity.

[0005] Meanwhile, existing oil palm harvesting equipment largely relies on manual visual assessment of fruit ripeness. Harvesting accuracy is heavily influenced by operator experience, easily leading to the accidental picking of immature fruit or the omission of ripe fruit. This not only reduces fruit quality but also increases unnecessary repetitive work costs. Current technologies that deeply integrate visual inspection technology with the oil palm fruit harvesting process are still limited, and there is a lack of complete mechanized solutions capable of effectively identifying ripe fruit and guiding harvesting operations. In the post-harvest collection and transportation stages, some existing devices still require manual assistance for fruit transfer or unloading, resulting in inefficient workflow and impacting overall operational efficiency. This mismatch between fruit collection and transportation heights is particularly pronounced under high-altitude harvesting conditions, further increasing operational complexity.

[0006] Therefore, there is an urgent need for an oil palm fruit harvesting machine that is compact in structure, has a wide range of adjustable working height, good stability, and can combine visual detection and environmental perception technologies to achieve precise harvesting of mature fruits. It should also have autonomous positioning and navigation capabilities, and take into account the efficient collection and transportation of fruits. This would improve the mechanization and intelligence of oil palm fruit harvesting operations, reduce labor intensity and safety risks, and meet the actual needs of large-scale and intensive oil palm production. Summary of the Invention

[0007] To address the shortcomings of existing technologies, the purpose of this invention is to provide a triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion positioning. This robot solves problems such as the limited height adjustment range of existing harvesting machinery, high safety risks due to reliance on manual assistance, low mechanization and discontinuous operation, inconsistent and vague standards for judging fruit maturity, and mismatch between fruit collection and transportation heights. The invention aims to achieve accurate judgment of the maturity of fruits from tall oil palm trees and precise harvesting over a wide range of height directions, thereby improving the efficiency of oil palm fruit collection and transportation and enhancing the mechanization and automation of oil palm fruit harvesting.

[0008] To solve the above problems, the technical solution provided by this invention patent is as follows: A triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization includes: a walking system, a fruit collection system, a picking system and a control system; The walking system includes a walking device and a support device. The walking device is used to move the whole machine, and the support device is used to support the ground during the harvesting operation to ensure the stability of the whole machine. The fruit collection system is installed on the walking system and includes a lifting device, a tilting device, and an information collection device. The lifting device is used to adjust the overall height of the fruit collection system, the tilting device is used to realize fruit transfer, and the information collection device is used for orchard positioning and image information collection and inspection. The harvesting system is installed on the walking system and includes a rotating device, a telescopic collecting device, and an end harvesting device. The rotating device is used to realize the rotation of the harvesting system, the telescopic collecting device is used to adjust the harvesting height and working distance, and the end harvesting device is used to identify and harvest mature fruits and complete the placement and collection.

[0009] The control system, as the central electrical control unit, is integrated inside the robot and is electrically connected to the walking device, fruit collection system, and harvesting system. It is used to receive signals uniformly, issue control commands centrally, and control the entire process of machine walking and parking, work support leveling, fruit box lifting and unloading, mechanical arm posture fine adjustment, and end-harvesting collaborative operation in a closed-loop linkage.

[0010] Furthermore, the traveling device includes a main body, a sub-body, main and sub-body slewing and telescopic hydraulic cylinders, triangular tracks, hydraulic motors, and a grille gate. The main body and sub-body are formed into a box structure by welding steel plates onto the frame. The main body box structure is enclosed, housing the power system, hydraulic system, and electronic control system. The main and sub-body slewing and telescopic hydraulic cylinders are located between the main body and sub-body to achieve vehicle length adjustment and posture adaptation. The hydraulic motors are located on both sides of the main body and sub-body. The triangular tracks are connected to the main body and sub-body respectively via hydraulic motors. The grille gate is connected to the main body via hinges and can be opened or closed to facilitate inspection and maintenance of the internal machinery. The main and sub-body slewing and telescopic hydraulic cylinders are connected between the main body and sub-body and can rotate and extend to achieve vehicle length adjustment and posture adaptation. The hydraulic motors are located on both sides of the main body and sub-body, and the triangular tracks are connected to the main body and sub-body respectively via hydraulic motors.

[0011] Furthermore, the support device includes a support leg, a support leg pivot, a support leg rotation cylinder, a support leg telescopic cylinder, and a support leg end. The support leg pivot is connected to both sides of the main body and the sub-body via bearings and bearing seats, respectively, and can rotate the support leg within a horizontal angle range under external drive. One end of the support leg is connected to the support leg pivot, and the other end is connected to the support leg end. The support leg rotation cylinder is used to drive the support leg pivot to rotate, thereby extending or retracting the support leg. The support leg telescopic cylinder is located between different sections of the support leg and is used to adjust the horizontal length of the support leg. The support leg end is located at the end of the support leg and is extended or retracted by a hydraulic cylinder to increase the vertical support length of the support device.

[0012] Furthermore, the lifting device is a scissor lift platform, including a load-bearing platform, a scissor linkage, a lifting hydraulic cylinder, a hydraulic cylinder hinge seat, a guide groove, sliding rollers, and a base. The load-bearing platform is connected to the end of the scissor linkage via the sliding rollers and the guide groove. The scissor linkage is positioned between the load-bearing platform and the base. The lifting hydraulic cylinder is connected to the scissor linkage via the hydraulic cylinder hinge seat and is used to drive the load-bearing platform to move up and down. The base is welded and fixed to the top of the main vehicle body and is connected to the end of the scissor linkage via the sliding rollers and the guide groove. The load-bearing platform, the scissor linkage, and the base together form a three-layer structure of the scissor lift platform. The sliding rollers and the guide groove limit the degrees of freedom so that the movement of the scissor lift platform is restricted to the vertical direction. The lifting hydraulic cylinder is connected to the scissor linkage via the hydraulic cylinder hinge seat and is used to drive the load-bearing platform to move up and down.

[0013] Furthermore, the tilting device includes a bearing seat, a tilting hydraulic cylinder, a fruit collection box, a fruit collection box baffle, and a fixed port; the front end of the fruit collection box is connected to the supporting platform via the bearing seat, and the rear end is connected to the tilting hydraulic cylinder via a hinge; the tilting hydraulic cylinder is connected to the supporting platform via the bearing seat and tilts the fruit by driving the rear end of the fruit collection box to tilt the fruit; the fruit collection box baffle is connected to and fixed to the fruit collection box via a hinge; the fixed port is fixed to the fruit collection box by welding.

[0014] Furthermore, the information acquisition device includes a navigation receiver, a multi-stage telescopic hydraulic cylinder, a camera bracket, and inspection cameras. The inspection cameras are respectively mounted on the camera bracket and face different directions, continuously acquiring environmental image data. Simultaneously, an inertial measurement unit is used to acquire the overall machine attitude and motion status information. Real-time pose estimation is achieved through the fusion of visual and inertial information, and an orchard environment map is constructed. The navigation receiver is fixed to the top of the fruit collection box, and global positioning information is fused and corrected to improve the global consistency and positioning accuracy of the map. The camera bracket is fixed to the top of the multi-stage telescopic hydraulic cylinder and is lifted by the multi-stage telescopic hydraulic cylinder. The lower end of the multi-stage telescopic hydraulic cylinder is fixed to the support platform by bolts.

[0015] Furthermore, the rotating device includes a harvesting robotic arm rotating base, which is fixed to the auxiliary vehicle body to realize the rotational movement of the harvesting system.

[0016] Furthermore, the telescopic collection device includes a lower multi-stage telescopic robotic arm, a pipeline protection hinge frame, a robotic arm angle adjustment cylinder, an upper multi-stage telescopic robotic arm, a multi-stage telescopic collection rigid pipe, a synchronization bracket, a collection pipe funnel opening, and a collection hose. The lower multi-stage telescopic robotic arm is bolted to the upper flange surface of the harvesting robotic arm's rotating base. The lower multi-stage telescopic robotic arm has three stages, each of which can extend and retract under the action of an internal hydraulic cylinder, thus achieving a large-area coverage in the vertical direction. The pipeline protection hinge frame is bolted to both the lower and upper multi-stage telescopic robotic arms, used for positioning and protecting the collection hose. The robotic arm angle adjustment cylinder is connected at both ends to both the lower and upper multi-stage telescopic robotic arms, used for adjusting the angle of the upper multi-stage telescopic robotic arm and providing height support. The upper multi-stage telescopic robotic arm extends and retracts via a hydraulic cylinder to adjust the working distance of the end-harvesting device. The multi-stage telescopic collection rigid pipe is connected to the upper multi-stage telescopic robotic arm via a synchronous bracket. The hydraulic cylinders inside the upper multi-stage telescopic robotic arm extend and retract synchronously at the corresponding stages, ensuring pipeline continuity during collection. The funnel opening of the collection pipe is connected to one end of the multi-stage telescopic collection rigid pipe, located near the harvesting shovel of the end harvesting device. The collected fruit can enter the funnel opening of the collection pipe by being tilted by the harvesting shovel. One end of the collection hose is connected to the multi-stage telescopic collection rigid pipe via an angled bend, and the other end is connected to a fixed port, forming a complete transportation route from the picking end to the collecting end, allowing the picked fruit to be transported to the fruit collection box via the collection pipeline.

[0017] Furthermore, the end-harvesting device includes a harvesting shovel, a harvesting camera, a harvesting shovel telescopic guide rail, a harvesting shovel telescopic hydraulic cylinder, a harvesting shovel directional plate, a harvesting shovel radial rotation cylinder, a harvesting shovel arc rotation guide rail, and a harvesting shovel pitch adjustment hinge. The harvesting shovel is fixedly connected to the harvesting shovel directional plate via the harvesting shovel telescopic guide rail and the harvesting shovel telescopic hydraulic cylinder. The path of the harvesting shovel is restricted by the harvesting shovel telescopic guide rail. The extension and retraction of the harvesting shovel hydraulic cylinder provides working power to the harvesting shovel, which drives the harvesting of mature fruits. The harvesting camera is located on both sides of the outer bottom surface of the harvesting shovel and is equipped with a vision system, which is responsible for visually detecting and identifying mature oil palm fruits. The harvesting shovel orientation plate serves as an intermediate structure between the harvesting shovel and the attitude adjustment system, as well as the attitude reference for the harvesting shovel, integrating its axial telescopic motion and multi-angle spatial motion. The radial rotation cylinder of the harvesting shovel connects the harvesting shovel orientation plate and the harvesting shovel arc rotation guide rail, enabling radial rotation when the required shovel angle is required. The harvesting shovel arc rotation guide rail is directly hinged to the harvesting shovel pitch adjustment mechanism, used to control the position adjustment of the harvesting shovel orientation plate within the plane of the harvesting shovel arc rotation guide rail, enabling it to work around trees without adjusting other large structures. The harvesting shovel pitch adjustment mechanism is hinged to the top of the upper multi-stage telescopic robotic arm, used to control the pitch angle adjustment of the harvesting shovel orientation plate in the vertical plane. Through these three structures, multi-angle position adjustment of the harvesting shovel is achieved.

[0018] Compared with other existing technologies, the present invention has the following advantages: 1. By incorporating a stable and reliable lifting device into a self-propelled tracked walking system and combining it with a multi-stage telescopic harvesting system, the harvesting system can achieve a wide range of continuous height adjustments in the vertical direction. The harvesting height can be adjusted to match the distribution location of the oil palm fruit. Compared to existing harvesting machinery with fixed or limited adjustment ranges, this invention effectively expands the harvesting system's operational adaptability in the height direction, meeting the harvesting needs of oil palm fruit at heights of approximately 5–15 m. Simultaneously, it reduces manual labor, eliminates the safety risks associated with high-altitude manual intervention, and avoids the problems of difficulty in harvesting high-altitude fruit or low operational efficiency caused by limited operating height.

[0019] 2. By coordinating the design of the lifting structure, the telescopic robotic arm, and the vision-based harvesting system, the end-harvesting device can accurately identify and harvest mature oil palm fruits at appropriate locations. The vision detection combines depth information acquisition with a deep learning-based maturity assessment method, and improves recognition stability in complex canopy occlusion environments through multi-frame target tracking and information fusion. This reduces reliance on human experience, lowers the incidence of mis-harvesting and missed harvests, and improves the consistency and accuracy of mature fruit identification. Simultaneously, the harvested fruits can be directly transported via collection pipelines to a liftable fruit collection box, and then transferred using a tilting device, effectively solving the problems of mismatched fruit collection and transfer heights and uncoordinated operations in existing technologies.

[0020] 3. This invention combines visual and inertial information fusion positioning and mapping technology to enable robots to autonomously locate and plan paths in complex oil palm orchard environments. By fusing with satellite navigation positioning information, it improves the positioning accuracy and motion stability during the overall operation, enabling the equipment to maintain continuous and efficient autonomous operation capabilities even in undulating terrain and tree-covered environments. This further enhances the efficiency and continuity of oil palm fruit picking and collection operations, as well as the intelligence and automation level of harvesting machinery. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall structure of the present invention. Figure 1 Figure 2 This is a schematic diagram of the overall structure of the present invention. Figure 2 Figure 3 This is a schematic diagram of the overall structure of the present invention. Figure 3 Figure 4 This is a schematic diagram of the walking system structure of the present invention. Figure 5 This is a schematic diagram of the fruit collection system structure of this invention patent. Figure 6 This is a schematic diagram of the harvesting system structure of the present invention. Reference numerals: 100-Walking system, 200-Fruit collection system, 300-Harvesting system, 110-Walking device, 120-Supporting device, 210-Lifting device (shearing lifting platform), 220-Tilting device, 230-Information collection device, 310-Rotating device, 320-Telescopic collection device, 330-End harvesting device, 111-Main vehicle body, 112-Sub-vehicle body, 113-Main and sub-vehicle body rotation telescopic hydraulic system 114-Cylinder, 115-Triangular track, 116-Hydraulic motor, 121-Grate gate, 122-Support leg, 123-Support leg rotating shaft, 124-Support leg telescopic cylinder, 125-Support leg end, 211-Load-bearing platform, 212-Scissor linkage, 213-Lifting hydraulic cylinder, 214-Hydraulic cylinder hinge seat, 215-Guide groove, 216-Sliding roller, 217-Base 221-Bearing seat, 222-Tilting hydraulic cylinder, 223-Fruit collection box, 224-Fruit collection box baffle, 225-Fixed port, 231-Navigation receiver, 232-Multi-stage telescopic cylinder, 233-Camera bracket, 234-Inspection camera, 311-Harvesting robotic arm rotating base, 321-Lower multi-stage telescopic robotic arm, 322-Pipeline protection hinge frame, 323-Robot arm angle adjustment cylinder, 324-Upper Multi-stage telescopic robotic arm, 325-multi-stage telescopic collection rigid tube, 326-synchronous support, 327-collection tube funnel opening, 328-collection hose, 331-harvesting shovel, 332-harvesting camera, 333-harvesting shovel telescopic guide rail, 334-harvesting shovel telescopic hydraulic cylinder, 335-harvesting shovel directional plate, 336-harvesting shovel radial rotation cylinder, 337-harvesting shovel arc rotation guide rail, 338-harvesting shovel pitch adjustment hinge. Detailed Implementation

[0022] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the following embodiments are given for illustrative purposes only and are not intended to limit the scope of the present invention. Those skilled in the art can make various modifications and substitutions to the present invention without departing from its spirit and essence.

[0023] like Figure 1 , 2 As shown in Figure 3, a triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion positioning includes a walking system, a fruit collection system, a picking system, and a control system.

[0024] like Figure 1 , 2 As shown in Figures 3 and 4, the walking system 100 is used to enable the machine to move and support operations within the oil palm orchard. The walking system 100 includes a walking device 110 and a support device 120.

[0025] The walking device 110 includes a main body 111, a secondary body 112, a main and secondary body slewing and telescopic hydraulic cylinder 113, a triangular track 114, a hydraulic motor 115, and a grille door 116. Both the main body 111 and the auxiliary body 112 are box-shaped structures formed by welding steel plates onto the frame. The main body 111 is a closed box structure, which houses the power system, hydraulic system, and electronic control system. The grille door 116 is connected to the main body 111 via hinges. The auxiliary body 112 is manufactured in the same way as the main body 111 and has a semi-open structure. The main and auxiliary body rotary telescopic cylinders 113 are located between the main body 111 and the auxiliary body 112 and can rotate and telescopically extend. The hydraulic motors 115 are respectively located on both sides of the main body 111 and the auxiliary body 112. The triangular tracks 114 are connected to the main body 111 and the auxiliary body 112 via the hydraulic motors 115, thereby realizing the tracked movement of the entire machine.

[0026] The support device 120 includes a support leg 121, a support leg rotating shaft 122, a support leg rotating cylinder 123, a support leg telescopic cylinder 124, and a support leg end 125. The support leg rotating shaft 122 is connected to both sides of the main body 111 and the sub-body 112 via bearings and bearing seats; the support leg rotating cylinder 123 is used to drive the support leg rotating shaft 122 to rotate, thereby extending or retracting the support leg 121; one end of the support leg 121 is connected to the support leg rotating shaft 122, and the other end is connected to the support leg end 125, and the support leg 121 adopts a two-stage telescopic structure; the support leg telescopic cylinder 124 is arranged between different stages of the support leg 121 and is used to adjust the length of the support leg 121 in the horizontal direction; the support leg end 125 is located at the end of the support leg 121 and is extended or retracted by a hydraulic cylinder to increase the support height of the support device 120 in the vertical direction.

[0027] The control system, as the central electrical control unit, is integrated inside the robot and is electrically connected to the walking device 110, the fruit collection system 200, and the picking system 300. It is used to uniformly receive signals, centrally issue control commands, and control the entire process of the robot's walking and parking, work support leveling, fruit box lifting and unloading, mechanical arm posture fine adjustment, and end-point picking collaborative operation in a closed-loop linkage.

[0028] like Figure 1 , 2 As shown in Figures 3 and 5, the fruit collection system 200 is installed on the walking system 100 and includes a lifting device 210, a tilting device 220, and an information collection device 230.

[0029] The lifting device 210 is a scissor lift platform, including a support platform 211, a scissor link 212, a lifting hydraulic cylinder 213, a hydraulic cylinder hinge seat 214, a guide groove 215, sliding rollers 216, and a base 217. The support platform 211 is connected to the end of the scissor link 212 through the sliding rollers 216 and the guide groove 215; the scissor link 212 is located between the support platform 211 and the base 217; the hydraulic cylinder hinge seat 214 is directly welded to the upper and lower opposite sides of the scissor link 212; the base 217 is welded and fixed to the top of the main body 111, and is connected to the end of the scissor link 212 through the sliding rollers 216 and the guide groove 215; the support platform 211, the scissor link 212, and the base 217 together form a three-layer structure of the scissor lift platform, with the degrees of freedom limited by the sliding rollers 216 and the guide groove 215. The lifting hydraulic cylinder 213 is connected to the scissor rod 212 via the hydraulic cylinder hinge seat 214.

[0030] The tilting device 220 includes a bearing seat 221, a tilting hydraulic cylinder 222, a fruit collection box 223, a fruit collection box baffle 224, and a fixing port 225. The front end of the fruit collection box 223 is connected to the support platform 211 via the bearing seat 221, and the rear end is connected to the tilting hydraulic cylinder 222 via a hinge. The tilting hydraulic cylinder 222 is connected to the support platform 211 via the bearing seat 221 and tilts the fruit by driving the rear end of the fruit collection box 223. The fruit collection box baffle 224 is connected to and fixed to the fruit collection box 223 via a hinge. The fixing port 225 is fixed to the fruit collection box 223 by welding.

[0031] The information collection device 230 includes a navigation receiver 231, a multi-stage telescopic cylinder 232, a camera bracket 233, and inspection cameras 234. The navigation receiver 231 is fixed to the upper end of the fruit collection box 223; the inspection cameras 234 are respectively installed on the camera bracket 233 and face four directions; the camera bracket 233 is fixed to the top of the multi-stage telescopic cylinder 232, and the inspection cameras 234 are positioned at the height of the fruit layer for detection by raising the multi-stage telescopic cylinder 232; the lower end of the multi-stage telescopic cylinder 232 is fixed to the support platform 211 by bolts.

[0032] like Figure 1 , 2 As shown in Figures 3 and 6, the harvesting system 300 is installed on the walking system 100 and includes a rotating device 310, a telescopic harvesting device 320, and an end harvesting device 330.

[0033] The rotating device 310 includes a harvesting robot arm rotating base 311, which is fixed inside the sub-vehicle body 112.

[0034] The telescopic collection device 320 includes a lower multi-stage telescopic robotic arm 321, a pipeline protection hinge frame 322, a robotic arm angle adjustment cylinder 323, an upper multi-stage telescopic robotic arm 324, a multi-stage telescopic collection rigid pipe 325, a synchronization bracket 326, a collection pipe funnel opening 327, and a collection hose 328. The lower multi-stage telescopic robotic arm 321 is bolted to the upper flange of the harvesting robotic arm rotating base 311. The pipeline protection hinge frame 322 is bolted to both the lower multi-stage telescopic robotic arm 321 and the upper multi-stage telescopic robotic arm 324, and is used to position and protect the collection hose 328. The robotic arm angle adjustment cylinder 323 is connected at both ends to the lower multi-stage telescopic robotic arm 321 and the upper multi-stage telescopic robotic arm 324. The upper multi-stage telescopic robotic arm 324 is connected to the pipeline protection hinge frame 322 and also includes three stages, controlled by internal hydraulic pressure. The cylinder extends and retracts to adjust the working distance of the end harvesting device 330; the multi-stage telescopic collection rigid pipe 325 is connected to the upper multi-stage telescopic robotic arm 324 through the synchronous bracket 326, and the hydraulic cylinder in the upper multi-stage telescopic robotic arm 324 drives synchronous extension and retraction in the corresponding stage; the collection pipe funnel opening 327 is connected to one end of the multi-stage telescopic collection rigid pipe 325, and its position is close to the harvesting shovel 331 of the end harvesting device 330; one end of the collection hose 328 is connected to the multi-stage telescopic collection rigid pipe 325 through an angled bend, and the other end is connected to the fixed port 225.

[0035] The end-harvesting device 330 includes a harvesting shovel 331, a harvesting camera 332, a harvesting shovel telescopic guide rail 333, a harvesting shovel telescopic hydraulic cylinder 334, a harvesting shovel orientation plate 335, a harvesting shovel radial rotation cylinder 336, a harvesting shovel arc rotation guide rail 337, and a harvesting shovel pitch adjustment hinge 338. The harvesting shovel 331 is fixedly connected to the harvesting shovel orientation plate 335 via the harvesting shovel telescopic guide rail 333 and the harvesting shovel telescopic hydraulic cylinder 334. The harvesting shovel telescopic guide rail 333 restricts the movement path of the harvesting shovel 331, and the extension and retraction of the harvesting shovel telescopic hydraulic cylinder 334 provides working power for the harvesting shovel 331. The harvesting camera 332 is located on both sides of the outer bottom surface of the harvesting shovel 331 and is used for visual inspection of oil palm fruits and identification of mature fruits. The harvesting shovel orientation plate 335 serves as the connection between the harvesting shovel 331 and the attitude adjustment system. The intermediate structure and attitude reference are used to integrate the axial telescopic movement and multi-angle spatial movement of the harvesting shovel 331; the radial rotation cylinder 336 of the harvesting shovel is connected to the harvesting shovel orientation plate 335 and the harvesting shovel arc rotation guide rail 337; the harvesting shovel arc rotation guide rail 337 is connected to the harvesting shovel pitch adjustment hinge 338; the harvesting shovel pitch adjustment hinge 338 is connected to the top of the upper multi-stage telescopic robotic arm 324 and the end of the harvesting shovel arc rotation guide rail 337, and the multi-angle position adjustment of the harvesting shovel 331 is realized through the above structure.

[0036] The working process of this invention patent is as follows: Before inspection, the overall machine status is checked, including the working status of the walking system 100, fruit collection system 200, harvesting system 300, and control system. It is also confirmed that all hydraulic actuators, sensors, and visual inspection devices are in usable condition. Upon arrival in the orchard, the multi-stage telescopic cylinder 232 extends and retracts, adjusting the inspection camera 234 to the optimal inspection and observation position. While the machine is inspecting within the orchard, the inspection camera 234 in the information acquisition device 230 continuously collects environmental image data. Simultaneously, the inertial measurement unit collects the machine's attitude and motion status information. Real-time pose estimation is achieved through the fusion of visual and inertial information, and an orchard environmental map is constructed. This relevant information is then transmitted back to the control system. This mapping process employs a simultaneous positioning and mapping method, continuously extracting environmental feature information during movement and performing matching and optimization to form an environmental model reflecting the orchard terrain, tree distribution, and obstacle information. Simultaneously, it combines this with global positioning information provided by the navigation receiver 231 for fusion and correction, improving the global consistency of the map and positioning accuracy. The constructed environmental map adopts a gridded representation, dividing the work area into passable areas and obstacle areas, and marking the location of the trees to realize the spatial positioning and numbering management of mature oil palm trees, providing a basis for subsequent fixed-point harvesting and route planning.

[0037] When harvesting is required, the machinery also needs to be checked for condition. Guided by the fused positioning results, the walking system 100 performs autonomous navigation. The control system uses the navigation receiver 231 as the receiving device, and integrates the information collected by the environmental perception and attitude measurement of the inspection camera 234 and the IMU in the control system to complete autonomous navigation and perform path planning, and controls the walking system 100 to drive stably according to the planned path.

[0038] Combining the grid map constructed during the inspection phase with the mature tree calibration information, the control system generates the optimal driving route through a path planning algorithm, enabling stable movement within the oil palm orchard. The navigation receiver 231 provides global position information for path correction and error correction, thereby improving the overall stability and positioning accuracy of the machine under complex terrain conditions. During operation, the lifting device 210 (scissor lift platform) in the fruit collection system 200, as well as the telescopic collection device 320 and the end harvesting device 330 in the harvesting system 300, are all in a non-operating state, positioned at their lowest possible level to reduce the overall height of the machine and facilitate movement between trees. Once the walking system 100 reaches the designated location of a mature oil palm tree, the machine stops in a relatively stable area accessible to multiple trees. Subsequently, the support device 120 unfolds, and the support leg shaft 122 is rotated by the support leg rotation cylinder 123 to unfold the support leg 121. The support leg extension cylinder 124 adjusts the length of the support leg 121 in the horizontal direction. The end 125 of the support leg extends and retracts through an internal hydraulic cylinder to increase the vertical support height of the support device 120. Finally, through the coordinated action of the internal components of the support device, the end 125 of the support leg makes reliable contact with the ground, thereby providing stable support for the entire machine.

[0039] At the start of the harvesting operation, the control system drives the harvesting system 300 to adjust its height and posture according to the target tree height. First, it extends the lower multi-stage telescopic robotic arm 321, raising the overall height of the upper working components, ultimately raising the end-harvesting device 330 to near the fruit distribution height. Then, the robotic arm angle adjustment cylinder 323 changes the working angle of the upper multi-stage telescopic robotic arm 324 and provides support during operation. In conjunction with the extension and retraction of the upper multi-stage telescopic robotic arm 324, it adjusts the working distance between the end-harvesting device 330 and the mature fruit. Once the end-harvesting device 330 approaches the target area, the harvesting camera 332 continuously captures images of the oil palm fruit. The vision system first preprocesses the captured images, including illumination compensation and feature enhancement. Then, it uses a deep learning-based visual recognition model to detect fruit clusters and determine their maturity, combining binocular vision to acquire fruit depth information and calculate the spatial three-dimensional coordinates of the mature fruit. To improve recognition stability, the system performs multi-frame target tracking and information fusion on detected fruit clusters in consecutive image frames. False detections are eliminated based on temporal consistency, and stable target position information is output. Based on the recognition results, the control system fine-tunes the parameters of the robotic arm angle adjustment cylinder 323 and the upper multi-stage telescopic robotic arm 324, ensuring the end-harvesting device 330 enters the appropriate harvesting area.

[0040] During the final posture fine-tuning stage, the radial angle of the harvesting shovel 331 is adjusted by the radial rotation cylinder 336, the horizontal position of the harvesting shovel 331 is adjusted by the arc-shaped rotary guide rail 337, and the vertical angle of the harvesting shovel 331 is adjusted by the pitch adjustment hinge 338. Through the coordinated action of these structures, the harvesting shovel 331 achieves a suitable shoveling posture. After posture adjustment, the harvesting shovel telescopic hydraulic cylinder 334 drives the harvesting shovel 331 to extend along the telescopic guide rail 333, cutting the stem of the mature oil palm fruit and retaining the fruit within the harvesting shovel 331. Subsequently, the final harvesting device 330 needs to be kept away from the tree to maintain a safe distance. Then, the harvesting shovel 331 tilts, guiding the fruit into the funnel opening 327 of the collection tube. The fruit is then transported to the collection box 223 via a complete transport route consisting of a multi-stage telescopic collection rigid tube 325, a collection flexible tube 328, and a fixed port 225, from the picking end to the collection end.

[0041] After the fruit is placed, the harvesting shovel 331 approaches the tree again. With minor adjustments made by the lower multi-stage telescopic robotic arm 321, the robotic arm angle adjustment cylinder 323, and the upper multi-stage telescopic robotic arm 324, the end harvesting device 330 inspects the tree. The harvesting camera 332 continuously acquires images and detects maturity of the current tree and its surrounding area, and updates the detection results based on the established tree spatial location model. If there are mature fruits, the above identification and harvesting process is repeated. If there are no mature fruits on the current tree, the grid map generated during the inspection phase and the mature tree marking information are used to determine whether there are other mature trees within the reachable area, and the location is adjusted and harvesting continues through autonomous path planning. When there are no more ripe fruits within the safe operating range, the robotic arm angle adjustment cylinder 323, the upper multi-stage telescopic robotic arm 324, and the lower multi-stage telescopic robotic arm 321 are sequentially returned to their original positions; the support device 120 is retracted; the walking system 100 is restarted, and the whole machine moves to the next picking marker position, repeating the above picking operation process.

[0042] During the harvesting process, when the number of fruits in the fruit collection box 223 reaches the preset capacity threshold, the control system issues a transfer command. The walking system 100 moves the entire machine to the preset unloading position. The control system determines whether the lifting device 210 needs to be driven to adjust the height based on the unloading height requirement. If so, the lifting hydraulic cylinder 213 drives the scissor linkage 212 to open through the hydraulic cylinder hinge seat 214. The sliding roller 216 and guide groove 217 limit the degrees of freedom so that the movement of the scissor lifting platform is restricted to the vertical direction, ultimately driving the carrying platform 211 to move up and down. After reaching the unloading area, the tilting device 220 is activated, and the tilting hydraulic cylinder 222 drives the rear end of the fruit collection box 223 to lift, opening the fruit collection box baffle 224 and unloading the fruits to the designated position. After unloading is completed, the tilting device 220 and the lifting device 210 are reset, and the entire machine returns to the breakpoint position to continue the harvesting operation.

[0043] When the harvesting task is completed in the work area, or when the energy level falls below the set threshold, the control system sequentially performs the return operation on the harvesting system 300, the fruit collection system 200, and the support device 120. Then, the walking system 100 combines the current position and the environmental grid map information to generate a return route through path planning, and returns to the machine room along the planned route under the guidance of the fused positioning results, completing the oil palm fruit harvesting operation and entering the maintenance stage.

[0044] The above embodiments are only used to illustrate the present invention and are not intended to limit the present invention. Those skilled in the art can make various changes and modifications without departing from the essence and scope of the present invention. Therefore, all equivalent technical solutions also fall within the protection scope of the present invention.

[0045] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

Claims

1. A triangular-tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization, characterized in that, include: Walking system (100), fruit collection system (200), harvesting system (300) and control system; The walking system (100) includes a walking device (110) and a support device (120). The walking device (110) is used to move the whole machine, and the support device (120) is used to support the ground during the harvesting operation to ensure the stability of the whole machine. The fruit collection system (200) is installed on the walking system (100) and includes a lifting device (210), a tilting device (220) and an information collection device (230). The lifting device (210) is used to adjust the overall height of the fruit collection system, the tilting device (220) is used to realize the transfer of fruit, and the information collection device (230) is used for orchard positioning and image information collection and inspection. The harvesting system (300) is installed on the walking system (100) and includes a rotating device (310), a telescopic collection device (320) and an end harvesting device (330). The rotating device (310) is used to realize the rotation of the harvesting system, the telescopic collection device (320) is used to adjust the harvesting height and working distance, and the end harvesting device (330) is used to identify and harvest mature fruits and complete the placement and collection. The control system, as the central electrical control unit, is integrated inside the robot and is electrically connected to the walking device (110), the fruit collection system (200), and the picking system (300). It is used to receive signals in a unified manner, issue control commands centrally, and control the entire process of the machine's walking and parking, work support leveling, fruit box lifting and unloading, mechanical arm posture fine adjustment, and end-picking collaborative operation in a closed-loop linkage.

2. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The walking device (110) includes a main body (111), a secondary body (112), a main and secondary body slewing and telescopic hydraulic cylinder (113), a triangular track (114), a hydraulic motor (115), and a grille door (116). The main and secondary body slewing and telescopic hydraulic cylinder (113) is located between the main body (111) and the secondary body (112) to realize the adjustment of the body length and posture adaptation. The hydraulic motor (115) is located on both sides of the main body (111) and the secondary body (112). The triangular track (114) is connected to the main body (111) and the secondary body (112) respectively through the hydraulic motor (115). The grille door (116) is connected to the main body (111) through a hinge and can be opened or closed to facilitate the inspection and maintenance of the internal machinery.

3. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The support device (120) includes a support leg (121), a support leg pivot (122), a support leg rotation cylinder (123), a support leg telescopic cylinder (124), and a support leg end (125); the support leg pivot (122) is connected to the main body (111) and the sub-body (112) on both sides via bearings and bearing seats and is rotatable; one end of the support leg (121) is connected to the support leg pivot (122), and the other end is connected to the support leg end (125). 25) Connection; The support leg rotating cylinder (123) is used to drive the support leg rotating shaft (122) to rotate, thereby enabling the support leg (121) to unfold or retract; The support leg telescopic cylinder (124) is set between different sections of the support leg (121) to adjust the horizontal length of the support leg (121); The support leg end (125) is set at the end of the support leg (121) and is extended and retracted by a hydraulic cylinder to increase the support length of the support device in the vertical direction.

4. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The lifting device (210) is a scissor lift platform, including a carrying platform (211), a scissor link (212), a lifting hydraulic cylinder (213), a hydraulic cylinder hinge seat (214), a guide groove (215), a sliding roller (216), and a base (217). The carrying platform (211) is connected to the end of the scissor link (212) through the sliding roller (216) and the guide groove (215). The scissor link (212) is located between the carrying platform (211) and the base (217). The lifting hydraulic cylinder (213) is connected to the scissor link (212) through the hydraulic cylinder hinge seat (214) and is used to drive the carrying platform (211) to move up and down. The base (217) is welded and fixed to the top of the main body (111) and is connected to the end of the scissor link (212) through the sliding roller (216) and the guide groove (215).

5. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The tilting device (220) includes a bearing seat (221), a tilting hydraulic cylinder (222), a fruit collection box (223), a fruit collection box baffle (224), and a fixed port (225). The front end of the fruit collection box (223) is connected to the support platform (211) through the bearing seat (221), and the rear end is connected to the tilting hydraulic cylinder (222) through a hinge. The tilting hydraulic cylinder (222) is connected to the support platform (211) through the bearing seat (221) and tilts the fruit by driving the rear end of the fruit collection box (223). The fruit collection box baffle (224) is connected to and fixed to the fruit collection box (223) through a hinge. The fixed port (225) is fixed to the fruit collection box (223) by welding.

6. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The information acquisition device (230) includes a navigation receiver (231), a multi-stage telescopic cylinder (232), a camera bracket (233), and an inspection camera (234). The inspection cameras (234) are installed on the camera bracket (233) and face different directions. They can continuously collect environmental image data and collect the attitude and motion status information of the whole machine using an inertial measurement unit. Real-time pose estimation is achieved by fusing visual and inertial information, and an orchard environment map is constructed. The navigation receiver (231) is fixed to the upper end of the fruit collection box (223). The global positioning information is fused and corrected to improve the global consistency and positioning accuracy of the map. The camera bracket (233) is fixed to the top of the multi-stage telescopic cylinder (232) and is lifted by the multi-stage telescopic cylinder (232). The lower end of the multi-stage telescopic cylinder (232) is fixed to the bearing platform (211) by bolts.

7. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The rotating device (310) includes a harvesting arm rotating base (311), which is fixed inside the sub-vehicle body (112).

8. The triangular tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The telescopic collection device (320) includes a lower multi-stage telescopic robotic arm (321), a pipeline protection hinge frame (322), a robotic arm angle adjustment cylinder (323), an upper multi-stage telescopic robotic arm (324), a multi-stage telescopic collection rigid pipe (325), a synchronization bracket (326), a collection pipe funnel opening (327), and a collection hose (328); the lower multi-stage telescopic robotic arm (321) is fixed to the upper flange surface of the harvesting robotic arm rotating base (311) by bolts; the pipeline protection hinge frame (322) is separated by bolts. The lower multi-stage telescopic robotic arm (321) and the upper multi-stage telescopic robotic arm (324) are connected to each other to position and protect the collection hose (328); the two ends of the robotic arm angle adjustment cylinder (323) are connected to the lower multi-stage telescopic robotic arm (321) and the upper multi-stage telescopic robotic arm (324) respectively to adjust the angle of the upper multi-stage telescopic robotic arm (324) and provide height support; the upper multi-stage telescopic robotic arm (324) extends and retracts through a hydraulic cylinder to adjust the working distance of the end harvesting device (330); The multi-stage telescopic collection tube (325) is connected to the upper multi-stage telescopic robotic arm (324) via a synchronous bracket (326), and achieves synchronous telescopic extension and retraction in the corresponding stages; the collection tube funnel opening (327) is connected to one end of the multi-stage telescopic collection tube (325); one end of the collection hose (328) is connected to the multi-stage telescopic collection tube (325), and the other end is connected to the fixed port (225), so that the harvested fruit is transported to the fruit collection box (223) through the collection pipeline.

9. A triangular-tracked self-propelled palm fruit harvesting robot based on visual perception and fusion localization according to claim 1, characterized in that: The end-harvesting device (330) includes a harvesting shovel (331), a harvesting camera (332), a harvesting shovel telescopic guide rail (333), a harvesting shovel telescopic hydraulic cylinder (334), a harvesting shovel directional plate (335), a harvesting shovel axial rotation cylinder (336), a harvesting shovel arc rotation guide rail (337), and a harvesting shovel pitch adjustment hinge (338). The harvesting shovel (331) is fixedly connected to the harvesting shovel directional plate (335) through the harvesting shovel telescopic guide rail (333) and the harvesting shovel telescopic hydraulic cylinder (334), and under its drive, it realizes the shoveling of mature fruits; the harvesting camera (332) A vision system is provided for visually detecting and identifying mature oil palm fruits; the harvest shovel orientation plate (335) serves as the attitude reference of the harvest shovel (331) and is radially rotated by the harvest shovel radial rotation cylinder (336); the harvest shovel arc rotation guide rail (337) is used to control the position adjustment of the harvest shovel orientation plate (335) within a 180° range in the plane; the harvest shovel pitch adjustment hinge (338) is used to control the pitch angle adjustment of the harvest shovel orientation plate (335) in the vertical plane, thereby realizing the multi-angle position adjustment of the harvest shovel (331).

Images