Pineapple picking and transporting integrated robot based on imaging technology

Abstract

The pineapple picking and transporting integrated robot based on the imaging technology comprises a walking frame, collecting mechanisms, mechanical arms and an end effector, the collecting mechanisms are hung on the two sides of the walking frame, and the four sets of mechanical arms are installed in the walking frame through rack grooves; the mechanical arm comprises a mounting box, a translation mechanism, a distance sensor, a hydraulic telescopic rod and an angle adjusting mechanism, the extended translation mechanism is mounted in the mounting box, and an end effector is mounted at the bottom of the mounting box through a sliding rod. Real-time tracking data of a camera and pineapple image data are fused together through a picking navigation system and are displayed on a display screen in a control panel in a three-dimensional graph mode, and a moving route from an end effector to a pineapple bottom end connecting handle and a moving route from the end effector to a pineapple crown are dynamically obtained. It is ensured that the navigation end effector accurately reaches the cutting point position, and the pineapple picking completeness is ensured.

Description

Technical Field

[0001] The invention relates to the technical field of agricultural equipment, and in particular to a pineapple harvesting and transportation integrated robot based on imaging technology. Background Art

[0002] The pineapple planting area in my country has increased year by year. At present, pineapples are mainly picked manually. Due to the rapid development of industrial production, a large number of agricultural laborers have been diverted, and the aging of the population has intensified. As a result, the number of workers engaged in agricultural production has become smaller and smaller. Manual labor alone can no longer meet the existing needs. Pineapple picking operations are facing the contradiction between the increasingly serious labor shortage and the rapid increase in costs. There are sharp thorns on the surface of pineapples, and manual picking can easily cause hand injuries to workers. In addition, the efficiency of pineapple picking is low, which will reduce economic benefits and affect income. With the development of computational image processing technology, it has become possible to use robots to pick pineapples. For this reason, a pineapple picking and transportation robot based on imaging technology is proposed.

[0003] In the process of realizing the present invention, the inventors found that at least the following problems exist in the prior art that have not been solved: the inability to dynamically navigate the cutting route of the end effector, which causes the cutting point position of the end effector to be easily offset when moving, causing damage to the pineapples during picking and increasing the loss rate of pineapple picking; at the same time, it is also impossible to cut the pineapple crown according to picking needs, which will increase the pineapple transportation space and increase the subsequent pineapple processing steps. For this reason, we propose a pineapple picking and transportation robot based on imaging technology to solve the existing problems. Summary of the invention

[0004] The purpose of the present invention is to solve the shortcomings existing in the prior art, such as: the inability to dynamically navigate the cutting route of the end effector, which causes the cutting point position of the end effector to be easily offset when moving, and the inability to cut the pineapple crown according to picking needs. The pineapple crown will increase the pineapple transportation space and increase the subsequent pineapple processing steps.

[0005] To achieve the above object, the present invention provides the following technical solutions: A pineapple harvesting and transportation robot based on imaging technology comprises a traveling frame, a collecting mechanism, a mechanical arm and an end effector, wherein the collecting mechanism is hoisted on both sides of the traveling frame, four sets of mechanical arms are installed inside the traveling frame through rack grooves, the mechanical arms comprise a mounting box, a translation mechanism, a distance sensor, a hydraulic telescopic rod, and an angle adjustment mechanism, an extended translation mechanism is installed inside the mounting box, and a distance sensor is embedded and installed on one side of the mounting box, and an end effector is installed at the bottom of the mounting box through a slide bar; The end effector includes a mounting frame, an axle rod, a cross-axle frame, a clamping and cutting claw, a hydraulic rod, and a camera. The axle rod is installed at the bottom of the mounting frame through an axle bolt, the clamping and cutting claw is fixedly installed on the inner side of the axle rod, the hydraulic rod is fixedly installed inside the mounting frame, the output end of the hydraulic rod is installed with a cross-axle frame through an axle bolt, one end of the cross-axle frame is movably connected to the inner side of the axle rod through the axle bolt, and the camera is fixedly installed on the front side of the walking frame.

[0006] Preferably, a control mechanism is fixedly mounted on the front of the traveling frame, and a control panel is embedded and mounted on the front of the control mechanism.

[0007] Preferably, walking mechanisms are fixedly installed on both sides of the rear end of the walking frame, and the walking mechanism includes a fixed frame, a drive motor, an encoder, and a moving wheel. The drive motor is fixedly installed on one side of the fixed frame, the moving wheel is fixedly installed on the output end of the drive motor, and the encoder is fixedly installed on one side of the drive motor.

[0008] Preferably, the collecting mechanism includes a frame, a collecting frame, a threaded hanger, and a threaded turntable, and two collecting frames are movably placed inside the frame, threaded hangers are installed on both sides of the frame through mounting sleeves, a threaded turntable is installed on the outer threads of the threaded hanger, and the threaded turntable consists of a thread sleeve and a mounting turntable, and the mounting turntable is installed on the top of the walking frame by bolts.

[0009] Preferably, a hydraulic telescopic rod is fixedly installed on the top of the mounting box, and the output end of the hydraulic telescopic rod is connected to the top of the mounting frame through a shaft sleeve. An angle adjustment mechanism is installed on the back of the mounting frame through an axle bolt, and the angle adjustment mechanism is connected to the output end of the hydraulic telescopic rod through the mounting sleeve.

[0010] Preferably, a steering mechanism is installed on both sides of the front end of the walking frame through axle bolts, and the steering mechanism includes a bogie, a dual-axis motor, a worm, and a worm wheel. A worm wheel is fixedly installed inside one end of the bogie, and a worm is fixedly installed at both ends of the dual-axis motor, and the worm is embedded in the worm wheel, and the dual-axis motor is fixedly installed inside the control mechanism.

[0011] Preferably, the first degree of freedom of the robotic arm is the lifting degree of freedom, which is executed by a hydraulic telescopic rod; the second degree of freedom is the horizontal degree of freedom, which is executed by a translation mechanism; and the third degree of freedom is the angle degree of freedom, which is executed by an angle adjustment mechanism.

[0012] Preferably, the control mechanism has a built-in visual recognition system and a picking navigation system, and the visual recognition system performs visual recognition and positioning through a training model.

[0013] Preferably, the camera performs visual recognition to determine the cutting point of the pineapple, the robotic arm drives the end effector to the cutting point, the clamping cutting claws clamp the pineapple and cut the pineapple connecting handle, and after cutting, the translation mechanism transports the pineapple to a pineapple collection frame.

[0014] Preferably, the visual recognition is achieved by the following steps: (1) Pre-store a large number of pineapple and pineapple crown images, build a pineapple image library, and perform deep learning. Apply the trained model to the camera's real-time image or video stream to identify and locate pineapples in real time; (2) Determine whether the pineapple fruit is blocked by stems and leaves. Obtain the pineapple point cloud information through the camera and determine whether the pineapple fruit is blocked by stems and leaves. If the pineapple fruit is not blocked by stems and leaves, determine whether the pineapple is tilted. If the pineapple fruit is blocked by stems and leaves, calculate the point cloud information of the pineapple crown and the pineapple fruit and the center of mass. Draw a straight line through the two center of mass points. The center of mass of the pineapple fruit exists on the straight line. Obtain the outermost point cloud information of the pineapple. (3) Calculate the coordinates of the bottom and crown of the pineapple, and reconstruct the three-dimensional surface of the pineapple fruit blocked by the stems and leaves through a 3D reconstruction unit based on the acquired point cloud information. This is achieved through computer vision and geometric modeling technology. The Poisson surface reconstruction algorithm is used to extract the surface mesh from the segmented volume data to obtain a three-dimensional model of the pineapple. The three-dimensional model of the pineapple is used for shape analysis and volume calculation. (4) Determine whether the pineapple is tilted. Use the Floyd algorithm to calculate the shortest distance between the coordinates of the bottom of the pineapple and the coordinates of the crown of the pineapple. Then calculate the center of the shortest distance. The center point of the shortest distance is the center of mass. The minimum distance from the bottom of the pineapple to the center of mass is connected by a line. If the inclination angle of the line connecting the line and the ground is less than 20°, the end effector reaches the cutting point for clamping and cutting. If the inclination angle is greater than 70°, the angle adjustment mechanism is used to drive the end effector to adjust the angle before clamping and cutting. (5) Real-time navigation: The picking navigation system integrates the real-time tracking data with the pineapple image data to perform real-time navigation to determine the shortest distance for the end effector to reach the cutting point.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The present invention combines real-time tracking data with pineapple image data through a picking navigation system, and displays the data in a three-dimensional graphic form on a display screen in a control panel. The visual information is used to navigate the shortest distance for the end effector to reach the cutting point in real time, and the moving route from the end effector to the connecting handle at the bottom of the pineapple and the moving route from the end effector to the pineapple crown are dynamically acquired, thereby ensuring that the navigation end effector accurately reaches the cutting point position, solving the problem of being unable to dynamically navigate the cutting route of the end effector, avoiding the easy displacement of the cutting point position when the end effector moves, ensuring the completeness of pineapple picking, and reducing the loss of pineapple picking. rate; at the same time, the order of pineapple cutting points is preset through the control panel, such as first locating the pineapple crown cutting point and then locating the pineapple bottom connecting handle cutting point, so that the device first cuts the pineapple crown and then cuts the pineapple bottom connecting handle, realizing flow operation, and removing the pineapple crown is beneficial to pineapple transportation and subsequent processing; the control mechanism controls multiple groups of end effectors to run simultaneously to pick and collect pineapples, realizing the function of multi-arm picking, thereby improving the picking speed and picking success rate, and reducing the cost of robot automated harvesting; at the same time, the maturity of the pineapple is identified and spatially positioned through the visual recognition system, thereby ensuring the quality of the picked fruit and work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is an overall schematic diagram of the present invention; Figure 2 It is a schematic diagram of the local structure of the walking mechanism of the present invention; Figure 3 It is a schematic diagram of the partial structure of the collecting mechanism of the present invention; Figure 4 It is a schematic diagram of the local structure of the connection between the robot arm and the end effector of the present invention; Figure 5 It is a schematic diagram of the local structure of the mechanical arm of the present invention; Figure 6 It is a schematic diagram of the local structure of the end effector of the present invention; Figure 7 It is a schematic diagram of the partial structure of the connection between the steering mechanism and the control mechanism of the present invention; Figure 8 It is a schematic diagram of the partial structure of the steering mechanism of the present invention; Fig. 9 The imaging technology flow chart of the present invention.

[0017] In the figure: 1. walking frame; 101. control mechanism; 102. control panel; 2. walking mechanism; 201. fixed frame; 202. driving motor; 203. encoder; 204. moving wheel; 3. collecting mechanism; 301. frame; 302. collecting frame; 303. threaded suspension rod; 304. threaded turntable; 4. robotic arm; 401. mounting box; 402. translation mechanism; 403. distance sensor; 404. hydraulic telescopic rod; 405. angle adjustment mechanism; 5. end effector; 501. mounting frame; 502. shaft rod; 503. cross-axis frame; 504. clamping and cutting claws; 505. hydraulic rod; 506. camera; 6. steering mechanism; 601. bogie; 602. dual-axis motor; 603. worm; 604. worm wheel. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] In the description of the present invention, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance.

[0020] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "provided with", "connected", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0021] See also Figure 1-9The present invention proposes a pineapple harvesting and transportation robot based on imaging technology, comprising a walking frame 1, a collecting mechanism 3, a mechanical arm 4 and an end effector 5. The collecting mechanism 3 is hoisted on both sides of the walking frame 1, and four sets of mechanical arms 4 are installed inside the walking frame 1 through rack grooves. The mechanical arm 4 includes a mounting box 401, a translation mechanism 402, a distance sensor 403, a hydraulic telescopic rod 404, and an angle adjustment mechanism 405. The installation box 401 is installed with an extended translation mechanism 402, and a distance sensor 403 is embedded and installed on one side of the installation box 401. The bottom of the installation box 401 is installed with an end effector 5 through a slide bar; The end effector 5 includes a mounting frame 501, an axis rod 502, a cross axis frame 503, a clamping and cutting claw 504, a hydraulic rod 505, and a camera 506. The axis rod 502 is installed at the bottom of the mounting frame 501 through an axis bolt, the clamping and cutting claw 504 is fixedly installed on the inner side of the axis rod 502, the hydraulic rod 505 is fixedly installed inside the mounting frame 501, the output end of the hydraulic rod 505 is installed with the cross axis frame 503 through an axis bolt, one end of the cross axis frame 503 is movably connected to the inner side of the axis rod 502 through an axis bolt, and the camera 506 is fixedly installed on the front side of the walking frame 1.

[0022] It should be noted that: the collecting mechanism 3 can follow and collect pineapples to achieve the integration of pineapple collection and collection. The translation mechanism 402 is composed of a servo motor, a transmission worm, a transmission worm wheel, a transmission gear set and a traveling gear. The output end of the servo motor is fixedly installed with a transmission worm, and a transmission worm wheel is installed on one side of the inner side of the installation box 401 through a bearing, and the transmission worm wheel is engaged with the transmission worm. The two ends of the worm wheel are installed with a transmission gear set through a connecting rod, and one side of the transmission gear set is fixedly installed with a traveling gear extending to the outside of the installation box 401. A rack groove meshing with the traveling gear is opened inside the traveling frame 1. The transmission worm is driven by the servo motor to drive the transmission worm wheel to rotate. The transmission worm wheel is connected to the transmission gear set to drive the traveling gear set to rotate, and the rotation is rotated inside the meshing rack groove to drive the end effector 5 to move horizontally. At the same time, the moving distance is monitored in real time by the distance sensor 403 to accurately control the horizontal moving position of the end effector 5. The camera 506 transmits the captured image to the control mechanism 101, which uses the built-in visual recognition system of the control mechanism 101 to visually identify and locate the position of the pineapple. The robotic arm 4 drives the end effector 5 to reach the cutting point, and the hydraulic rod 505 drives the cross-axis frame 503 to drive the shaft rod 502 to flip. The flipped shaft rod 502 drives the clamping and cutting claws 504 to open and close the clamping claws 504, and the cutting blade at the bottom center of the clamping and cutting claws 504 is used to clamp and cut the cutting point.

[0023] Furthermore, a control mechanism 101 is fixedly installed on the front of the traveling frame 1 , and a control panel 102 is embedded and installed on the front of the control mechanism 101 .

[0024] It should be noted that the control mechanism 101 is connected to the control panel 102, the walking mechanism 2, the robotic arm 4, the end effector 5 and the steering mechanism 6, and a transformer is placed inside the control mechanism 101, and the transformer is electrically connected to the two sets of drive motors 202. The order of pineapple cutting points is preset through the control panel 102, such as first locating the pineapple crown cutting point and then locating the pineapple bottom connecting handle cutting point, so that the device first cuts the pineapple crown and then cuts the pineapple bottom connecting handle, realizing assembly line operation.

[0025] Furthermore, walking mechanisms 2 are fixedly installed on both sides of the rear end of the walking frame 1. The walking mechanism 2 includes a fixed frame 201, a driving motor 202, an encoder 203, and a moving wheel 204. The driving motor 202 is fixedly installed on one side of the fixed frame 201, the moving wheel 204 is fixedly installed on the output end of the driving motor 202, and the encoder 203 is fixedly installed on one side of the driving motor 202.

[0026] It should be noted that: the driving motor 202 drives the moving wheel 204 to rotate, and the rotating moving wheel 204 drives the device to move independently in the pineapple field, realizing the functions of automated mobile picking and transportation. At the same time, the encoder 203 is used to detect the number of rotations of the moving wheel 204, and the calculation is performed based on the number of rotations of the moving wheel 204 and the moving path of the pineapple field, and the picking route of the device is designed in cooperation with the steering mechanism 6.

[0027] To add an explanation, when the pineapple field is slippery after rain, the mobile wheel 204 can be replaced with a crawler wheel, and the drive motor 202 drives the crawler wheel to rotate. The rotating crawler wheel drives the device to move automatically in the pineapple field, thereby realizing the functions of automated mobile picking and transportation, and the power supply voltage transmitted to the drive motor 202 is controlled by the transformer inside the control mechanism 101. The rotation speed of the two sets of crawler wheels is controlled by adjusting the power supply voltage, thereby achieving the purpose of separately controlling the rotation speed of the two sets of crawler wheels. The device turns and moves forward through crawler wheels with different speeds on both sides, and at the same time, the encoder 203 is used to detect the number of rotations of the crawler wheels, and the calculation is performed based on the number of rotations of the crawler wheels and the moving path of the pineapple field. The picking route of the device is designed through the crawler wheel steering design and the speed measurement of the encoder 203. The crawler wheels are convenient for walking in the land, and the device can walk smoothly in slippery land.

[0028] Furthermore, the collecting mechanism 3 includes a frame 301, a collecting frame 302, a threaded hanging rod 303, and a threaded turntable 304, and two collecting frames 302 are movably placed inside the frame 301, and the threaded hanging rods 303 are installed on both sides of the frame 301 through mounting sleeves, and the threaded turntable 304 is threadedly installed on the outer side of the threaded hanging rods 303, and the threaded turntable 304 consists of a thread sleeve and a mounting turntable, and the mounting turntable is installed on the top of the walking frame 1 by bolts.

[0029] It should be noted that: rotating the threaded sleeve drives the threaded hanging rod 303 to perform telescopic adjustment, and the position height of the collection frame 302 inside the frame 301 is adjusted by adjusting the extension length of the threaded hanging rod 303, so as to facilitate the adjustment according to the height of the staff to take the collection frame 302.

[0030] Furthermore, a hydraulic telescopic rod 404 is fixedly installed on the top of the mounting box 401, and the output end of the hydraulic telescopic rod 404 is connected to the top of the mounting frame 501 through a shaft sleeve. An angle adjustment mechanism 405 is installed on the back of the mounting frame 501 through an axle bolt, and the angle adjustment mechanism 405 is connected to the output end of the hydraulic telescopic rod 404 through the mounting sleeve.

[0031] It should be noted that the hydraulic telescopic rod 404 drives the end effector 5 to move up and down, and the angle adjustment mechanism 405 is composed of an electric rod and an axis connecting rod. When the electric rod drives the axis connecting rod to move the axis, the end effector 5 is driven to adjust the angle. Furthermore, a steering mechanism 6 is installed on both sides of the front end of the walking frame 1 through axle bolts, and the steering mechanism 6 includes a bogie 601, a dual-axis motor 602, a worm 603, and a worm wheel 604. The worm wheel 604 is fixedly installed inside one end of the bogie 601, and the worm 603 is fixedly installed at both ends of the dual-axis motor 602. The worm 603 is embedded in the worm wheel 604, and the dual-axis motor 602 is fixedly installed inside the control mechanism 101.

[0032] It should be noted that the dual-axis motor 602 drives the worm 603 to rotate, and the rotating worm 603 drives the bogie 601 to rotate through the worm wheel 604, thereby adjusting the steering of the bogie 601 and cooperating with the walking mechanism 2 to realize automatic turning movement of the device.

[0033] Furthermore, the first degree of freedom of the robot arm 4 is the lifting degree of freedom, which is performed by the hydraulic telescopic rod 404; the second degree of freedom is the horizontal degree of freedom, which is performed by the translation mechanism 402; the third degree of freedom is the angle degree of freedom, which is performed by the angle adjustment mechanism 405; It should be noted that the first degree of freedom is mainly used to lift the end effector 5, the second degree of freedom drives the end effector 5 to move horizontally, and the third degree of freedom drives the end effector 5 to adjust the angle. According to the control instructions of the control mechanism 101, the end effector 5 is sent to the position of the working pineapple to realize the picking of the pineapple.

[0034] Furthermore, the control mechanism 101 has a built-in visual recognition system and a picking navigation system, and the visual recognition system performs visual recognition and positioning through a training model.

[0035] It should be noted that: a deep learning model is constructed to analyze the real-time image or video stream taken by the camera 506, identify the maturity of the pineapple for identification and spatial positioning, and establish a unified coordinate system through the picking navigation system to associate the three-dimensional model data of the pineapple, the unit pixel distance data of the camera 506 and the position of the end effector 5.

[0036] Further, the camera 506 performs visual recognition to determine the cutting point of the pineapple, the robotic arm 4 drives the end effector 5 to reach the cutting point, the clamping claws 504 clamp the pineapple and cut the pineapple connecting handle, and after cutting, the translation mechanism 402 transports the pineapple to the pineapple collection frame 302.

[0037] It should be noted that the camera 506 transmits the captured image to the built-in visual recognition system and the picking navigation system of the control mechanism 101, and converts it to the world coordinate system through the camera 506 internal parameter matrix and external parameter matrix, so as to obtain the actual distance. The actual distance is divided by the pixel distance to obtain the unit pixel distance; Further, visual recognition is achieved through the following steps: (1) Pre-store a large number of pineapple and pineapple crown images, build a pineapple image library, and perform deep learning; build a pineapple image library 1) Data collection: Collect a large number of pineapple images, including pineapple images from different angles, different lighting conditions, and different backgrounds to ensure the generalization ability of the model; 2) Data labeling: Label the collected images and mark the location and category of the pineapple. This can be done by using professional labeling tools to generate corresponding label files; 3) Data preprocessing: Perform necessary preprocessing on the image, such as scaling and normalization, to facilitate model training; Building a deep learning model 1) Environment setup: Install a deep learning framework (such as PyTorch); 2) Model configuration: According to the task requirements, configure the relevant parameters of the model, such as input image size, anchor box size, etc.; 3) Model training: Use the prepared pineapple image library and label files to train the model. During the training process, the model will continuously learn how to identify pineapples and predict their locations. 4) Model evaluation: Evaluate the performance of the model on the validation set or test set, including indicators such as accuracy and recall; 5) Model optimization: Based on the evaluation results, optimize the model, such as adjusting the learning rate, increasing the number of training rounds, etc., to improve the performance of the model; Applying the trained model to the real-time image or video stream of camera 506 to identify and locate the pineapple in real time; (2) Determine whether the pineapple fruit is blocked by stems and leaves, obtain pineapple point cloud information through camera 506, determine whether the pineapple fruit is blocked by stems and leaves, if the pineapple fruit is not blocked by stems and leaves, determine whether the pineapple is tilted, if the pineapple fruit is blocked by stems and leaves, calculate the point cloud information of the pineapple crown and the pineapple fruit and calculate the center of mass, draw a straight line through the two center of mass points, the center of mass of the pineapple fruit exists on the straight line, and obtain the outermost point cloud information of the pineapple; (3) Calculate the coordinates of the bottom and crown of the pineapple, and reconstruct the three-dimensional surface of the pineapple fruit blocked by the stems and leaves through a 3D reconstruction unit based on the acquired point cloud information. This is achieved through computer vision and geometric modeling technology, and the Poisson surface reconstruction algorithm is used to extract the surface mesh from the segmented volume data. Firstly, the indicator function χM is solved by constructing the Poisson equation Δχ=∇⋅(∇χ)=∇⋅V→, where V→ is a vector field, which is approximated by the gradient of the sample points. The Poisson equation is solved by the finite element method to obtain the expression of the implicit function χ. Based on the corner value χ(p) of the voxel and the isovalue γ of the unknown object M, the MC algorithm is used to extract the isosurface from the voxel and output the triangular mesh to obtain the three-dimensional model of the pineapple. The three-dimensional model of the pineapple is used for shape analysis and volume calculation. Then, import the 3D model into professional 3D editing software (such as Maya, 3dsMax), extract the triangular face information of the model through the 3D editing software, including the vertex index and vertex coordinates of each triangular face, and obtain the coordinates of the bottom of the pineapple and the crown of the pineapple; (4) Determine whether the pineapple is tilted, calculate the shortest distance between the coordinates of the bottom end of the pineapple and the coordinates of the crown of the pineapple by using the Floyd algorithm, and then calculate the center of the shortest distance. The center point of the shortest distance is the center of mass. The minimum distance from the bottom end of the pineapple to the center of mass is connected by a line. If the inclination angle of the line connecting the line and the ground is less than 20°, the end effector 5 reaches the cutting point for clamping and cutting; if the inclination angle is greater than 70°, the angle adjustment mechanism 405 is used to drive the end effector 5 to adjust the angle for clamping and cutting; (5) Real-time navigation: The picking navigation system integrates the real-time tracking data with the pineapple image data to perform real-time navigation to determine the shortest distance for the end effector 5 to reach the cutting point, obtain the moving distance from the end effector 5 to the connecting handle at the bottom of the pineapple, and the moving distance from the end effector 5 to the pineapple crown, and ensure that the navigation end effector 5 accurately reaches the cutting point.

[0038] Working principle: The order of pineapple cutting points is preset through the control panel 102, such as first locating the pineapple crown cutting point, and then locating the pineapple bottom connecting handle cutting point, the driving motor 202 drives the moving wheel 204 to rotate, and the rotating moving wheel 204 drives the device to move in the pineapple field by itself, and at the same time, the double-axis motor 602 drives the worm 603 to rotate, and the rotating worm 603 drives the bogie 601 to rotate through the worm wheel 604, so as to adjust the steering of the bogie 601, cooperate with the walking mechanism 2 to realize the automatic turning movement of the device, and under the drive of the walking mechanism 2 and the steering mechanism 6, mobile picking is realized; The camera 506 transmits the captured image to the built-in visual recognition system and the picking navigation system of the control mechanism 101. The visual recognition system performs visual recognition, performs real-time recognition and spatial positioning of the pineapple, and then the system obtains the point cloud information of the pineapple to determine whether the pineapple fruit is blocked by stems and leaves. If the pineapple fruit is not blocked by stems and leaves, it is determined whether the pineapple is tilted. If the pineapple fruit is blocked by stems and leaves, the point cloud information of the pineapple crown and the pineapple fruit and the center of mass are obtained, and a straight line passing through the two center of mass points is made. The center of mass of the pineapple fruit exists on the straight line, and the outermost point cloud information of the pineapple is obtained. According to the obtained point cloud information, the 3D reconstruction unit is used to reconstruct the three-dimensional surface of the pineapple fruit blocked by stems and leaves. This is achieved through computer vision and geometric modeling technology. The surface mesh is extracted from the segmented volume data using the Poisson surface reconstruction algorithm, and the triangular mesh is output to obtain the three-dimensional model of the pineapple. The three-dimensional model is imported into professional 3D editing software (such as Maya, 3dsMax), and the triangular surface information of the model is extracted through the 3D editing software. The three-dimensional model of the pineapple is used for shape analysis and volume calculation to obtain the coordinates of the bottom end of the pineapple and the coordinates of the pineapple crown. According to the acquired coordinates of the bottom of the pineapple and the crown of the pineapple, the shortest distance between the coordinates of the bottom of the pineapple and the crown of the pineapple is connected, the center point of the shortest distance is taken as the centroid, and the minimum distance from the bottom of the pineapple to the centroid is connected. If the inclination angle of the straight line connecting the straight line and the ground is less than 20°, the robot arm 4 and the end effector 5 are directly executed to clamp and cut the identified pineapple. If the inclination angle is greater than 70°, the angle adjustment mechanism 405 is used to drive the end effector 5 to adjust the angle for clamping and cutting; The picking navigation system combines the real-time tracking data with the pineapple image data and displays them in a three-dimensional graphic form on the display screen in the control panel 102. The visual information is used to navigate the end effector 5 in real time to the shortest distance to the cutting point, ensuring that the navigation end effector 5 accurately reaches the cutting point position; The picking navigation system executes four groups of mechanical arms 4 and the end effector 5. The translation mechanism 402 drives the end effector 5 to translate, and the hydraulic telescopic rod 404 drives the end effector 5 to move up and down. When reaching the cutting point, the hydraulic rod 505 drives the cross-axis frame 503 to drive the shaft rod 502 to flip inward, and the inward-turned shaft rod 502 drives the clamping and cutting claws 504 to close, and the cutting blade at the bottom center of the clamping and cutting claws 504 is used to clamp and cut the cutting point to remove the pineapple crown, and then according to the preset sequence, the translation mechanism 402, the hydraulic telescopic rod 404 and the end effector 5 are started to cut the connecting handle at the bottom end of the pineapple. The cut pineapple is driven by the translation mechanism 402 to move the end effector 5 to the top of the collection frame 302, and the hydraulic rod 505 drives the cross-axis frame 503 to drive the shaft rod 502 to flip outward, and the outward-turned shaft rod 502 drives the clamping and cutting claws 504 to expand, and the pineapple is put into the collection frame 302, which is convenient for collection and transportation, and realizes the integration of pineapple transportation.

[0039] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A pineapple harvesting and transporting robot based on imaging technology, comprising a walking frame (1), a collecting mechanism (3), a mechanical arm (4) and an end effector (5), characterized in that: Collection mechanisms (3) are hoisted on both sides of the traveling frame (1); four sets of mechanical arms (4) are installed inside the traveling frame (1) through rack grooves; the mechanical arms (4) comprise a mounting box (401), a translation mechanism (402), a distance sensor (403), a hydraulic telescopic rod (404), and an angle adjustment mechanism (405); an extended translation mechanism (402) is installed inside the mounting box (401), and a distance sensor (403) is embedded and installed on one side of the mounting box (401); and an end effector (5) is installed at the bottom of the mounting box (401) through a sliding rod; The end effector (5) comprises a mounting frame (501), an axle rod (502), a cross-axis frame (503), a clamping and cutting claw (504), a hydraulic rod (505), and a camera (506), wherein the bottom of the mounting frame (501) is mounted with the axle rod (502) via an axle bolt, the inner side of the axle rod (502) is fixedly mounted with the clamping and cutting claw (504), the inner side of the mounting frame (501) is fixedly mounted with the hydraulic rod (505), the output end of the hydraulic rod (505) is mounted with the cross-axis frame (503) via an axle bolt, one end of the cross-axis frame (503) is movably connected to the inner side of the axle rod (502) via an axle bolt, and the front side of the walking frame (1) is fixedly mounted with the camera (506).

2. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: A control mechanism (101) is fixedly mounted on the front of the traveling frame (1), and a control panel (102) is embedded and mounted on the front of the control mechanism (101).

3. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: A walking mechanism (2) is fixedly mounted on both sides of the rear end of the walking frame (1); the walking mechanism (2) comprises a fixed frame (201), a drive motor (202), an encoder (203), and a moving wheel (204); the drive motor (202) is fixedly mounted on one side of the fixed frame (201); the moving wheel (204) is fixedly mounted on the output end of the drive motor (202); and the encoder (203) is fixedly mounted on one side of the drive motor (202).

4. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: The collecting mechanism (3) comprises a frame (301), a collecting frame (302), a threaded suspension rod (303), and a threaded turntable (304), wherein two collecting frames (302) are movably placed inside the frame (301), threaded suspension rods (303) are mounted on both sides of the frame (301) via mounting sleeves, a threaded turntable (304) is threadedly mounted on the outer side of the threaded suspension rod (303), and the threaded turntable (304) consists of a threaded sleeve and a mounting turntable, and the mounting turntable is mounted on the top of the traveling frame (1) via bolts.

5. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: A hydraulic telescopic rod (404) is fixedly installed on the top of the installation box (401) and penetrates through it. The output end of the hydraulic telescopic rod (404) is connected to the top of the installation frame (501) via a shaft sleeve. An angle adjustment mechanism (405) is installed on the back of the installation frame (501) via a shaft bolt, and the angle adjustment mechanism (405) is connected to the output end of the hydraulic telescopic rod (404) via the installation sleeve.

6. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: Steering mechanisms (6) are installed on both sides of the front end of the walking frame (1) via axle bolts. The steering mechanism (6) comprises a bogie (601), a dual-axis motor (602), a worm (603), and a worm wheel (604). The worm wheel (604) is fixedly installed inside one end of the bogie (601), the worm (603) is fixedly installed at both ends of the dual-axis motor (602), and the worm (603) and the worm wheel (604) are engaged. The dual-axis motor (602) is fixedly installed inside the control mechanism (101).

7. The pineapple harvesting and transporting robot based on imaging technology according to claim 1, characterized in that: The first degree of freedom of the mechanical arm (4) is a lifting degree of freedom, which is performed by a hydraulic telescopic rod (404); the second degree of freedom is a horizontal degree of freedom, which is performed by a translation mechanism (402); The third degree of freedom is the angle degree of freedom, which is implemented by the angle adjustment mechanism (405).

8. The pineapple harvesting and transporting robot based on imaging technology according to claim 2, characterized in that: The control mechanism (101) has a built-in visual recognition system and a picking navigation system, and the visual recognition system performs visual recognition and positioning through a training model.

9. A pineapple harvesting and transporting robot based on imaging technology according to claims 1-8, characterized in that: The camera (506) performs visual recognition to determine the cutting point of the pineapple, the robotic arm (4) drives the end effector (5) to reach the cutting point, the clamping and cutting claws (504) clamp the pineapple and cut the pineapple connecting handle, and after cutting is completed, the translation mechanism (402) transports the pineapple to the pineapple collection frame (302).

10. The pineapple harvesting and transporting robot based on imaging technology according to claim 9, characterized in that: The visual recognition is achieved by the following steps: (1) Pre-store a large number of pineapple and pineapple crown images, build a pineapple image library, and perform deep learning; (2) determining whether the pineapple fruit is blocked by stems and leaves, obtaining pineapple point cloud information through a camera (506), and determining whether the pineapple fruit is blocked by stems and leaves; (3) Calculate the coordinates of the bottom of the pineapple and the crown of the pineapple, and reconstruct the three-dimensional surface of the pineapple fruit blocked by the stems and leaves through a 3D reconstruction unit based on the acquired point cloud information, using computer vision and geometric modeling technology; (4) Determine whether the pineapple is tilted, calculate the shortest distance between the coordinates of the bottom of the pineapple and the coordinates of the crown of the pineapple using the Floyd algorithm, and then calculate the center of the shortest distance. The center point of the shortest distance is the center of mass. The minimum distance from the bottom of the pineapple to the center of mass is connected by a line. If the inclination angle of the line connecting the line and the ground is less than 20°, the end effector (5) reaches the cutting point for clamping and cutting; if the inclination angle is greater than 70°, the end effector (5) adjusts the angle before clamping and cutting; (5) Real-time navigation: The real-time tracking data of (506) is integrated with the pineapple image data through the picking navigation system to perform real-time navigation for the end effector (5) to reach the shortest distance of the cutting point.

Images