Pineapple picking and transporting integrated robot based on imaging technology

Abstract

The application discloses a pineapple picking and transporting integrated robot based on an imaging technology, which comprises a walking frame, a collecting mechanism, a mechanical arm and an end effector, the walking frame is provided with the collecting mechanism on both sides, the walking frame is internally provided with four groups of mechanical arms through a rack slot, the mechanical arm comprises a mounting box, a translation mechanism, a distance sensor, a hydraulic telescopic rod and an angle adjusting mechanism, the mounting box is internally provided with the extending translation mechanism, and the bottom of the mounting box is provided with the end effector through a sliding rod. The real-time tracking data of the camera and the pineapple image data are fused together through the picking navigation system, and are displayed in the form of three-dimensional graphics on the display screen in the control panel, the moving route of the end effector to the connecting handle at the bottom of the pineapple and the moving route of the end effector to the crown of the pineapple are dynamically acquired, the navigation end effector can accurately arrive at the cutting point position, and the pineapple picking completeness is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural equipment, in particular to a pineapple picking and transporting integrated robot based on imaging technology. BACKGROUND

[0002] The planting area of pineapple in China increases year by year, and at present, pineapple mainly relies on manual picking. Due to the rapid development of industrial production, a large number of agricultural labor forces are diverted, and the population aging is intensified, so that the labor force engaged in agricultural production is less and less. Manual labor cannot meet the current needs, and the pineapple picking operation is facing the contradiction between the increasingly serious labor shortage and the rapid rising cost. The pineapple surface has sharp thorns, and manual picking is easy to cause hand injury of the workers, and the pineapple picking efficiency is low, which will reduce the economic benefit and affect the income. With the development of computer image processing technology, it is possible to pick pineapple by robot. Therefore, a pineapple picking and transporting integrated robot based on imaging technology is proposed.

[0003] In the process of realizing the present application, the inventors found that at least the following problems in the prior art have not been solved: the end effector cannot dynamically navigate the cutting route, which causes the end effector to easily deviate from the cutting point position when moving, resulting in damage to the pineapple during picking and increasing the loss rate of pineapple picking; and the pineapple crown cannot be cut according to the picking needs, which increases the pineapple transportation space and subsequent pineapple processing procedures. Therefore, a pineapple picking and transporting integrated robot based on imaging technology is proposed to solve the existing problems. SUMMARY

[0004] The purpose of the present application is to solve the problems in the prior art, such as the inability to dynamically navigate the cutting route of the end effector, which causes the end effector to easily deviate from the cutting point position when moving, and the inability to cut the pineapple crown according to the picking needs, which increases the pineapple transportation space and subsequent pineapple processing procedures.

[0005] To achieve the above purpose, the present application provides the following technical scheme:

[0006] A pineapple picking and transporting integrated robot based on imaging technology, comprising a walking frame, a collecting mechanism, a mechanical arm and an end effector, the walking frame is provided with a collecting mechanism on both sides, the inside of the walking frame is provided with four groups of mechanical arms through a rack groove, the mechanical arm comprises a mounting box, a translation mechanism, a distance sensor, a hydraulic telescopic rod and an angle adjusting mechanism, the inside of the mounting box is provided with an extending translation mechanism, and a distance sensor is embedded on one side of the inside of the mounting box, and the bottom of the mounting box is provided with an end effector through a slide rod;

[0007] The end effector comprises a mounting frame, a shaft rod, a cross shaft frame, a clamping cutting claw, a hydraulic rod, and a camera, the bottom of the mounting frame is provided with the shaft rod through a shaft bolt, the inner side of the shaft rod is fixedly provided with the clamping cutting claw, the inside of the mounting frame is fixedly provided with the hydraulic rod, the output end of the hydraulic rod is provided with the cross shaft frame through a shaft bolt, and one end of the cross shaft frame is movably connected with the inner side of the shaft rod through a shaft bolt.

[0008] Preferably, the front surface of the walking frame is fixedly provided with a control mechanism, and the front surface of the control mechanism is embeddedly provided with a control panel.

[0009] Preferably, the two sides of the rear end of the walking frame are fixedly provided with walking mechanisms, the walking mechanism comprises a fixed frame, a driving motor, an encoder and a moving wheel, one side of the fixed frame is fixedly provided with the driving motor, the output end of the driving motor is fixedly provided with the moving wheel, and one side of the driving motor is fixedly provided with the encoder.

[0010] Preferably, the collecting mechanism comprises a frame, a collecting frame, a threaded lifting rod and a threaded turntable, two collecting frames are movably placed in the inside of the frame, the threaded lifting rod is installed on the two sides of the frame through a mounting sleeve, the threaded turntable is screwedly installed on the outer side of the threaded lifting rod, the threaded turntable is composed of a screw sleeve and a mounting turntable, and the mounting turntable is installed on the top of the walking frame through bolts.

[0011] Preferably, the top of the mounting box is fixedly provided with a penetrating hydraulic telescopic rod, the output end of the hydraulic telescopic rod is connected with the top of the mounting frame through a shaft sleeve, the back surface of the mounting frame is provided with an angle adjusting mechanism through a shaft bolt, and the angle adjusting mechanism is connected with the output end of the hydraulic telescopic rod through a mounting sleeve.

[0012] Preferably, the two sides of the front end of the walking frame are provided with steering mechanisms through shaft bolts, the steering mechanism comprises a steering frame, a double-shaft motor, a worm and a worm gear, the inside of one end of the steering frame is fixedly provided with the worm gear, the two ends of the double-shaft motor are fixedly provided with the worm, the worm is embedded with the worm gear, and the double-shaft motor is fixedly installed in the inside of the control mechanism.

[0013] Preferably, the first degree of freedom of the mechanical arm is a lifting degree of freedom, which is executed by the hydraulic telescopic rod, the second degree of freedom is a horizontal degree of freedom, which is executed by the translation mechanism, and the third degree of freedom is an angle degree of freedom, which is executed by the angle adjusting mechanism.

[0014] Preferably, the control mechanism is provided with a visual recognition system and a picking navigation system, and the visual recognition system performs visual recognition and positioning through a trained model.

[0015] Preferably, the camera performs visual recognition to determine the pineapple cutting point, the mechanical arm drives the end effector to the cutting point, the clamping jaw clamps the pineapple and cuts the pineapple connecting handle, and the translation mechanism transports the pineapple into the pineapple collection frame.

[0016] Preferably, the visual recognition is achieved by the following steps:

[0017] (1) A large number of pineapple and pineapple crown images are stored in advance to construct a pineapple image library, and deep learning is performed, and the trained model is applied to real-time image or video stream of the camera to realize real-time recognition and positioning of the pineapple;

[0018] (2) It is judged whether the pineapple fruit is covered by the stem and leaf, the point cloud information of the pineapple is obtained through the camera, it is judged whether the pineapple fruit is covered by the stem and leaf, if the pineapple fruit is not covered by the stem and leaf, it is judged whether the pineapple is inclined, if the pineapple fruit is covered by the stem and leaf, the point cloud information of the pineapple crown and the pineapple fruit is calculated, a straight line passing through the two mass center points is drawn, the mass center of the pineapple fruit exists on the straight line, and the outermost point cloud information of the pineapple is obtained;

[0019] (3) The pineapple bottom end coordinate and the pineapple crown coordinate are calculated, the pineapple fruit covered by the stem and leaf is reconstructed by a 3D reconstruction unit according to the obtained point cloud information, the reconstruction is realized through computer vision and geometric modeling technology, a surface mesh is extracted from the segmented volume data by using a Poisson surface reconstruction algorithm, a three-dimensional model of the pineapple is obtained, and shape analysis and volume calculation are performed on the three-dimensional model of the pineapple;

[0020] (4) It is judged whether the pineapple is inclined, the shortest distance between the pineapple bottom end coordinate and the pineapple crown coordinate is calculated by using the Floyd algorithm, then the center of the shortest distance is calculated, the center point of the shortest distance is the mass center, the minimum distance from the pineapple bottom end to the mass center is calculated, if the inclination angle between the straight line and the ground is less than 20°, the end effector reaches the cutting point for clamping and cutting; if the inclination angle is greater than 70°, the angle adjusting mechanism is used to drive the end effector to adjust the angle, and then clamping and cutting are performed;

[0021] (5) Real-time navigation, the real-time tracking data of the camera is fused with the pineapple image data through the picking navigation system, and the end effector reaches the cutting point in the shortest distance.

[0022] Compared with the prior art, the present application has the following advantages:

[0023] The present application fuses the real-time tracking data of the camera and the pineapple image data together through the picking navigation system, and displays on the display screen in the control panel in the form of three-dimensional graphics, and through these visual information, the shortest distance of the end effector to reach the cutting point is navigated in real time, the moving route of the end effector to the connecting handle at the bottom of the pineapple and the moving route of the end effector to the pineapple crown are dynamically obtained, the end effector is accurately navigated to the cutting point position, the problem that the end effector cannot be dynamically navigated to the cutting route is solved, the end effector is prevented from deviating from the cutting point position during movement, the integrity of the pineapple picking is ensured, and the loss rate of the pineapple picking is reduced; meanwhile, the cutting point sequence of the pineapple is preset through the control panel, such as positioning the cutting point of the pineapple crown first, and then positioning the cutting point of the connecting handle at the bottom of the pineapple, so that the device cuts the pineapple crown first and then cuts the connecting handle at the bottom of the pineapple, realizes the flow operation, and removing the pineapple crown is beneficial to the transportation and subsequent processing of the pineapple; the multiple end effectors are controlled to run simultaneously through the control mechanism to pick and collect the pineapple, the multi-arm picking function is realized, and therefore the picking speed and the picking success rate are improved, and the cost of the robot automatic harvesting is reduced; meanwhile, the maturity of the pineapple is identified and spatially positioned through the visual recognition system, and therefore the quality and the working efficiency of the picked fruit are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 It is a whole schematic view of the present application;

[0025] Figure 2 It is a local structure schematic view of the walking mechanism of the present application;

[0026] Figure 3 It is a local structure schematic view of the collecting mechanism of the present application;

[0027] Figure 4 It is a local structure schematic view of the mechanical arm and the end effector connection of the present application;

[0028] Figure 5 It is a local structure schematic view of the mechanical arm of the present application;

[0029] Figure 6 It is a local structure schematic view of the end effector of the present application;

[0030] Figure 7 It is a local structure schematic view of the turning mechanism and the control mechanism connection of the present application;

[0031] Figure 8 It is a local structure schematic view of the turning mechanism of the present application;

[0032] Figure 9 It is an imaging technology flow chart of the present application.

[0033] In the figure: 1, walking frame; 101, control mechanism; 102, control panel; 2, walking mechanism; 201, fixed frame; 202, drive motor; 203, encoder; 204, moving wheel; 3, collection mechanism; 301, frame; 302, collection frame; 303, threaded boom; 304, threaded turntable; 4, mechanical arm; 401, mounting box; 402, translation mechanism; 403, distance sensor; 404, hydraulic telescopic rod; 405, angle adjusting mechanism; 5, end effector; 501, mounting frame; 502, shaft rod; 503, cross shaft frame; 504, clamping jaw; 505, hydraulic rod; 506, camera; 6, steering mechanism; 601, steering frame; 602, double-shaft motor; 603, worm; 604, worm gear. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0035] In the description of the present application, it should be noted that the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end", "the other end" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, therefore cannot be understood as a limitation on the present application. In addition, the terms "first" and "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0036] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "provided with", "connection" and the like should be broadly understood, for example, "connection" can be fixed connection, can also be detachable connection, or integral connection; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through an intermediate medium, can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0037] EMBODIMENT

[0038] Please refer to Figures 1-9The pineapple picking and transporting integrated robot based on the imaging technology comprises a walking frame 1, a collecting mechanism 3, a mechanical arm 4 and an end effector 5, the collecting mechanism 3 is hung on both sides of the walking frame 1, four groups of mechanical arms 4 are installed in the walking frame 1 through a rack groove, the mechanical arm 4 comprises an installation box 401, a translation mechanism 402, a distance sensor 403, a hydraulic telescopic rod 404 and an angle adjusting mechanism 405, the translation mechanism 402 is installed in the installation box 401 and extends outwards, the distance sensor 403 is embedded and installed on one side in the installation box 401, and the end effector 5 is installed on the bottom of the installation box 401 through a sliding rod;

[0039] The end effector 5 comprises a mounting frame 501, a shaft rod 502, a cross shaft frame 503, a clamping cutting claw 504, a hydraulic rod 505 and a camera 506, the shaft rod 502 is installed on the bottom of the mounting frame 501 through a shaft bolt, the clamping cutting claw 504 is fixedly installed on the inner side of the shaft rod 502, the hydraulic rod 505 is fixedly installed in the mounting frame 501, the cross shaft frame 503 is installed on the output end of the hydraulic rod 505 through a shaft bolt, one end of the cross shaft frame 503 is movably connected with the inner side of the shaft rod 502 through a shaft bolt, and the camera 506 is fixedly installed on the front face of the walking frame 1.

[0040] It should be noted that the collecting mechanism 3 can follow and collect pineapples, realize the integration of pineapple collection and collection, the translation mechanism 402 is composed of a servo motor, a transmission worm, a transmission worm wheel, a transmission gear set and a walking gear, the output end of the servo motor is fixedly installed with the transmission worm, the transmission worm wheel is installed on one side in the installation box 401 through a bearing, and the transmission worm wheel is connected with the transmission worm in a clamping mode, the transmission gear set is installed on both ends of the transmission worm wheel through a connecting rod, one side of the transmission gear set is fixedly installed with the walking gear which extends to the outside of the installation box 401, the rack groove which is meshed with the walking gear is formed in the walking frame 1, the transmission worm is driven to rotate by the servo motor, the transmission worm wheel is driven to rotate by the transmission gear set through connection, the walking gear is driven to rotate by the transmission gear set, the rotating walking gear rotates and moves in the rack groove, and the end effector 5 is driven to move horizontally, and the distance sensor 403 is used for monitoring the moving distance in real time, so that the horizontal moving position of the end effector 5 is accurately controlled.

[0041] The camera 506 transmits the photographed pictures to the control mechanism 101, the pineapple position is positioned through the visual identification system built in the control mechanism 101, the mechanical arm 4 drives the end effector 5 to reach the cutting point, the hydraulic rod 505 drives the cross shaft frame 503 to drive the shaft rod 502 to overturn, the overturned shaft rod 502 drives the clamping cutting claw 504 to open and close, and the cutting blade at the bottom center of the clamping cutting claw 504 is used for clamping and cutting the cutting point.

[0042] Further, the front side of the walking frame 1 is fixedly installed with a control mechanism 101, and the front side of the control mechanism 101 is embeddedly installed with a control panel 102.

[0043] It should be noted that the control mechanism 101 is connected with the control panel 102, the walking mechanism 2, the mechanical arm 4, the end effector 5 and the steering mechanism 6, and a transformer is placed in the control mechanism 101, and the transformer is electrically connected with the two groups of driving motors 202, and the pineapple cutting point sequence is preset through the control panel 102, such as positioning the pineapple crown cutting point first, and then positioning the pineapple bottom end connecting handle cutting point, so that the device cuts the pineapple crown first and then cuts the pineapple bottom end connecting handle, realizing the flow operation.

[0044] Further, the two sides of the rear end of the walking frame 1 are fixedly installed with the walking mechanism 2, and the walking mechanism 2 comprises a fixed frame 201, a driving motor 202, an encoder 203 and a moving wheel 204, and one side of the fixed frame 201 is fixedly installed with the driving motor 202, the output end of the driving motor 202 is fixedly installed with the moving wheel 204, and one side of the driving motor 202 is fixedly installed with the encoder 203.

[0045] It should be noted that the driving motor 202 drives the moving wheel 204 to rotate, and the rotating moving wheel 204 drives the device to move in the pineapple field automatically, realizing the functions of automatic moving picking and transporting, and the encoder 203 is used to detect the rotating number of the moving wheel 204, and the picking route of the device is designed according to the rotating number of the moving wheel 204 and the moving path of the pineapple field.

[0046] It should be noted that when the pineapple field is wet and slippery after raining, the moving wheel 204 can be replaced with a tracked wheel, the driving motor 202 drives the tracked wheel to rotate, and the rotating tracked wheel drives the device to move in the pineapple field automatically, realizing the functions of automatic moving picking and transporting, and the transformer in the control mechanism 101 is used to control the power supply voltage to the driving motor 202, the rotating speed of the two groups of tracked wheels is controlled by adjusting the power supply voltage, the rotating speed of the two groups of tracked wheels is controlled respectively, the device advances by steering through the two sides of different rotating speed tracked wheels, the encoder 203 is used to detect the rotating number of the tracked wheel, and the picking route of the device is designed according to the rotating number of the tracked wheel and the moving path of the pineapple field, the tracked wheel is convenient for walking in the land, and the device can walk stably in the wet and slippery land.

[0047] Further, the collecting mechanism 3 comprises a frame 301, collecting frames 302, threaded hangers 303, threaded turntables 304, and two collecting frames 302 are movably arranged inside the frame 301, the threaded hangers 303 are installed on the two sides of the frame 301 through mounting sleeves, the threaded turntables 304 are installed on the outer sides of the threaded hangers 303 in a threaded mode, and the threaded turntable 304 is composed of a threaded sleeve and a mounting turntable, and the mounting turntable is installed on the top of the walking frame 1 through bolts.

[0048] It should be noted that the threaded sleeve drives the threaded hanger 303 connected in a threaded mode to be adjusted in extension and retraction, the position height of the collecting frame 302 inside the frame 301 is adjusted by adjusting the extension length of the threaded hanger 303, and the position height of the collecting frame 302 is adjusted according to the convenience of the height of the staff.

[0049] Further, the top of the mounting box 401 is fixedly installed with a penetrating hydraulic telescopic rod 404, the output end of the hydraulic telescopic rod 404 is connected with the top of the mounting frame 501 through a shaft sleeve, the back of the mounting frame 501 is installed with an angle adjusting mechanism 405 through a shaft bolt, and the angle adjusting mechanism 405 is connected with the output end of the hydraulic telescopic rod 404 through a mounting sleeve.

[0050] It should be noted that the hydraulic telescopic rod 404 drives the end effector 5 to move up and down, the angle adjusting mechanism 405 is composed of an electric rod and a shaft connecting rod, and the shaft connecting rod is driven by the electric rod to move the shaft to drive the end effector 5 to adjust the angle;

[0051] Further, the walking frame 1 is provided with a steering mechanism 6 on the two sides of the front end through a shaft bolt, the steering mechanism 6 comprises a steering frame 601, a double-shaft motor 602, a worm 603 and a worm gear 604, the worm gear 604 is fixedly installed in the inside of one end of the steering frame 601, the worm 603 is fixedly installed on the two ends of the double-shaft motor 602, the worm 603 is embedded with the worm gear 604, and the double-shaft motor 602 is fixedly installed in the inside of the control mechanism 101.

[0052] It should be noted that the double-shaft motor 602 drives the worm 603 to rotate, the rotating worm 603 drives the steering frame 601 to rotate through the worm gear 604, so as to adjust the steering of the steering frame 601, and the walking mechanism 2 is matched to realize the automatic turning movement of the device.

[0053] Further, the first degree of freedom of the mechanical arm 4 is the lifting degree of freedom, which is executed by the hydraulic telescopic rod 404; the second degree of freedom is the horizontal degree of freedom, which is executed by the translation mechanism 402; and the third degree of freedom is the angle degree of freedom, which is executed by the angle adjusting mechanism 405.

[0054] It should be noted that: the first degree of freedom is mainly to lift the end effector 5, the second degree of freedom drives the end effector 5 to move horizontally, and the third degree of freedom drives the end effector 5 to adjust the angle. According to the control instruction of the control mechanism 101, the end effector 5 is sent to the position of the working pineapple to realize the picking of the pineapple.

[0055] Further, the control mechanism 101 is built-in with a visual recognition system and a picking navigation system. The visual recognition system performs visual recognition and positioning through a trained model.

[0056] It should be noted that the deep learning model analyzes the real-time images or video streams captured by the camera 506, identifies the maturity of the pineapple for identification and spatial positioning, and establishes a unified coordinate system through the picking navigation system to associate the three-dimensional model data of the pineapple, the unit pixel distance data of the camera 506, and the position of the end effector 5.

[0057] Further, the camera 506 performs visual recognition to determine the cutting point of the pineapple, the mechanical arm 4 drives the end effector 5 to reach the cutting point, the clamping jaw 504 clamps the pineapple and cuts the connecting handle of the pineapple, and the translation mechanism 402 transports the pineapple to the pineapple collection frame 302 after cutting.

[0058] It should be noted that the camera 506 transmits the captured images to the visual recognition system and the picking navigation system built-in the control mechanism 101, and converts them to the world coordinate system through the intrinsic matrix and extrinsic matrix of the camera 506, so as to obtain the actual distance, which is divided by the pixel distance to obtain the unit pixel distance.

[0059] Further, the visual recognition is realized through the following steps:

[0060] (1) A large number of pineapple and pineapple crown images are stored in advance to construct a pineapple image library and perform deep learning;

[0061] Construction of pineapple image library

[0062] 1) Data collection: Collect a large number of pineapple images, including pineapple pictures under different angles, different lighting conditions, and different backgrounds, to ensure the generalization ability of the model;

[0063] 2) Data labeling: Label the collected images to mark the position and category of the pineapple. This can be done by using professional labeling tools to generate corresponding label files;

[0064] 3) Data preprocessing: Perform necessary preprocessing on the images, such as scaling and normalization, to facilitate model training;

[0065] Construction of deep learning model

[0066] 1) Environment setup: Install deep learning framework (e.g. PyTorch);

[0067] 2) Model configuration: Configure the relevant parameters of the model according to the task requirements, such as input image size, anchor box size, etc.;

[0068] 3) Model training: Train the model using the prepared pineapple image library and label file. During the training process, the model will continuously learn how to recognize and locate pineapples;

[0069] 4) Model evaluation: Evaluate the performance of the model on the validation set or test set, including accuracy, recall rate, etc.;

[0070] 5) Model optimization: Optimize the model according to the evaluation results, such as adjusting the learning rate, increasing the number of training rounds, etc., to improve the performance of the model;

[0071] Apply the trained model to the camera 506 real-time image or video stream to perform real-time recognition and positioning of pineapples;

[0072] (2) Determine whether the pineapple fruit is covered by the stem and leaves. Obtain the point cloud information of the pineapple through the camera 506, determine whether the pineapple fruit is covered by the stem and leaves, if the pineapple fruit is not covered by the stem and leaves, determine whether the pineapple is inclined, if the pineapple fruit is covered by the stem and leaves, find the point cloud information of the pineapple crown and the pineapple fruit, find the center of mass, draw a straight line through the two center points, there is a pineapple fruit center on the straight line, and obtain the outermost point cloud information of the pineapple;

[0073] (3) Calculate the pineapple bottom coordinate and pineapple crown coordinate. Reconstruct the three-dimensional surface of the pineapple fruit covered by the stem and leaves through the 3D reconstruction unit according to the obtained point cloud information, realize through computer vision and geometric modeling technology, use the Poisson surface reconstruction algorithm to extract the surface mesh from the segmented volume data,

[0074] First, solve the indicator function χM by constructing the Poisson equation where V→ is a vector field, which is approximated by the gradient of the sample points, and the Poisson equation is solved by the finite element method to obtain the expression of the implicit function χ, based on the corner point value χ(p) of the voxel and the unknown object M, the isosurface is extracted from the voxel using the MC algorithm, the triangular mesh is output, and the three-dimensional model of the pineapple is obtained, and the three-dimensional model of the pineapple is used for shape analysis and volume calculation;

[0075] Then, import the three-dimensional model into professional 3D editing software (such as Maya, 3dsMax), extract the triangular face information of the model through the 3D editing software, including the vertex index and vertex coordinates of each triangular face, obtain the pineapple bottom coordinate and pineapple crown coordinate;

[0076] (4) judge whether the pineapple is inclined, calculate the shortest distance between the pineapple bottom end coordinate and the pineapple crown coordinate by Floyd algorithm, then calculate the center of the shortest distance, the center point of the shortest distance is the centroid, the minimum distance from the pineapple bottom end to the centroid, if the inclination angle of the connecting line is less than 20°, the end effector 5 reaches the cutting point for clamping and cutting; if the inclination angle is greater than 70°, the angle adjusting mechanism 405 drives the end effector 5 to adjust the angle for clamping and cutting;

[0077] (5) real-time navigation, the camera 506 real-time tracking data is fused with the pineapple image data by the picking navigation system, the shortest distance of the end effector 5 reaching the cutting point is navigated in real time, the moving distance of the end effector 5 to the pineapple bottom end connecting handle and the moving distance of the end effector 5 to the pineapple crown are obtained, and the end effector 5 is ensured to accurately reach the cutting point position.

[0078] Working principle: the pineapple cutting point sequence is preset by the control panel 102, such as positioning the pineapple crown cutting point first, then positioning the pineapple bottom end connecting handle cutting point, the driving motor 202 drives the moving wheel 204 to rotate, the rotating moving wheel 204 drives the device to move in the pineapple field, at the same time, the double-shaft motor 602 drives the worm 603 to rotate, the rotating worm 603 drives the bogie 601 to rotate through the worm gear 604, so as to adjust the steering of the bogie 601, and realize the automatic turning movement of the device in cooperation with the walking mechanism 2, realize the mobile picking under the driving of the walking mechanism 2 and the steering mechanism 6.

[0079] The camera 506 transmits the photographed pictures to the visual recognition system and the picking navigation system in the control mechanism 101, the visual recognition system is used for visual recognition, real-time recognition and spatial positioning of the pineapple, then the system obtains the pineapple point cloud information, judges whether the pineapple fruit is blocked by the stem and leaf, if the pineapple fruit is not blocked by the stem and leaf, judges whether the pineapple is inclined, if the pineapple fruit is blocked by the stem and leaf, the point cloud information of the pineapple crown and the pineapple fruit is calculated, and the centroid is calculated, a straight line passing through the two centroid points is drawn, there is a pineapple fruit centroid on the straight line, the outermost point cloud information of the pineapple is obtained, the 3D reconstruction unit is used to reconstruct the three-dimensional surface of the pineapple fruit blocked by the stem and leaf according to the obtained point cloud information, which is realized by computer vision and geometric modeling technology, the surface mesh is extracted from the segmented volume data by using the Poisson surface reconstruction algorithm, the triangular mesh is output, the three-dimensional model of the pineapple is obtained, the three-dimensional model is imported into professional 3D editing software (such as Maya, 3dsMax), the triangular surface information of the model is extracted by the 3D editing software, the three-dimensional model of the pineapple is used for shape analysis and volume calculation, and the pineapple bottom end coordinate and the pineapple crown coordinate are obtained.

[0080] According to the obtained pineapple bottom end coordinates and pineapple crown coordinates, the shortest distance between the pineapple bottom end coordinates and the pineapple crown coordinates is connected, the center point of the shortest distance is taken as the centroid, the minimum distance between the pineapple bottom end and the centroid is connected, if the inclination angle of the connecting line with the ground line is less than 20°, the mechanical arm 4 and the end effector 5 directly perform clamping and cutting on the identified pineapple, if the inclination angle is greater than 70°, the angle adjusting mechanism 405 drives the end effector 5 to adjust the angle and then clamps and cuts;

[0081] The picking navigation system fuses the real-time tracking data and the pineapple image data together, and displays them in the form of three-dimensional graphics on the display screen in the control panel 102, and visually guides the end effector 5 to reach the shortest distance to the cutting point in real time, so as to ensure that the end effector 5 accurately reaches the cutting point position;

[0082] The picking navigation system executes four groups of mechanical arms 4 and end effectors 5, the translation mechanism 402 drives the end effector 5 to translate, the hydraulic telescopic rod 404 drives the end effector 5 to move up and down, when reaching the cutting point position, the hydraulic rod 505 drives the cross shaft frame 503 to drive the shaft rod 502 to turn inward, the inward turning shaft rod 502 drives the clamping and cutting jaw 504 to close, the cutting blade at the bottom center of the clamping and cutting jaw 504 clamps and cuts the cutting point, and then according to the preset sequence, the translation mechanism 402, the hydraulic telescopic rod 404 and the end effector 5 are started to cut the pineapple bottom end connecting handle, the pineapple after cutting is moved to above the collecting frame 302 by the end effector 5 driven by the translation mechanism 402, the hydraulic rod 505 drives the cross shaft frame 503 to drive the shaft rod 502 to turn outward, the outward turning shaft rod 502 drives the clamping and cutting jaw 504 to expand, and the pineapple is put into the collecting frame 302, so that the pineapple is conveniently collected and transported, and the pineapple picking and transporting integration is realized.

[0083] It is apparent to those skilled in the art that the present application is not limited to the details of the foregoing exemplary embodiments, and that the present application can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the present application being defined by the appended claims rather than the foregoing description, and it is intended to include all changes falling within the meaning and scope of equivalents of the claims. Any reference signs in the claims should not be construed as limiting the claims to which they belong.

Claims

1. An imaging technology-based pineapple picking and transporting integrated robot, comprising a walking frame (1), a collecting mechanism (3), a mechanical arm (4) and an end effector (5), characterized in that: The walking frame (1) is hoisted with a collection mechanism (3) on both sides, the inside of the walking frame (1) is installed with four groups of mechanical arms (4) through rack slot, the mechanical arm (4) includes installation box (401), translation mechanism (402), distance sensor (403), hydraulic telescopic rod (404), angle adjusting mechanism (405), the inside of the installation box (401) is installed with the translation mechanism (402) extending, and the inside of the installation box (401) is embedded with distance sensor (403) on one side, the bottom of the installation box (401) is installed with end effector (5) through slide bar; The end effector (5) includes mounting bracket (501), shaft rod (502), cross shaft bracket (503), clamping jaw (504), hydraulic rod (505), camera (506), and the bottom of the mounting bracket (501) is installed with the shaft rod (502) through the shaft bolt, the inner side of the shaft rod (502) is fixedly installed with the clamping jaw (504), the inside of the mounting bracket (501) is fixedly installed with the hydraulic rod (505), the output end of the hydraulic rod (505) is installed with the cross shaft bracket (503) through the shaft bolt, one end of the cross shaft bracket (503) is movably connected with the inner side of the shaft rod (502) through the shaft bolt, and the front of the walking frame (1) is fixedly installed with the camera (506).

2. The pineapple harvesting and transporting integrated robot based on imaging technology according to claim 1, characterized in that: The front of the walking frame (1) is fixedly installed with the control mechanism (101), and the front of the control mechanism (101) is embeddedly installed with the control panel (102).

3. The pineapple harvesting and transporting integrated robot based on imaging technology according to claim 1, characterized in that: The walking mechanism (2) is fixedly installed on both sides of the rear end of the walking frame (1), and the walking mechanism (2) includes fixed frame (201), driving motor (202), encoder (203) and moving wheel (204), one side of the fixed frame (201) is fixedly installed with the driving motor (202), the output end of the driving motor (202) is fixedly installed with the moving wheel (204), and one side of the driving motor (202) is fixedly installed with the encoder (203).

4. The pineapple harvesting and transporting integrated robot based on imaging technology according to claim 1, characterized in that: The collection mechanism (3) includes frame (301), collection frame (302), threaded lifting rod (303) and threaded turntable (304), two collection frames (302) are movably placed in the inside of the frame (301), threaded lifting rods (303) are installed on both sides of the frame (301) through mounting sleeves, threaded turntables (304) are screwedly installed on the outer sides of the threaded lifting rods (303), and the threaded turntable (304) is composed of a threaded sleeve and a mounting turntable, and the mounting turntable is installed on the top of the walking frame (1) through bolts.

5. The imaging technology-based pineapple harvesting and transporting integrated robot according to claim 1, characterized in that: The top of the installation box (401) is fixedly installed with the penetrating hydraulic telescopic rod (404), the output end of the hydraulic telescopic rod (404) is connected with the top of the mounting bracket (501) through a shaft sleeve, the back of the mounting bracket (501) is installed with the angle adjusting mechanism (405) through the shaft bolt, and the angle adjusting mechanism (405) is connected with the output end of the hydraulic telescopic rod (404) through the mounting sleeve.

6. The imaging technology-based pineapple harvesting and transporting integrated robot according to claim 1, characterized in that: The walking frame (1) is provided with a steering mechanism (6) on both sides of the front end through shaft bolts, the steering mechanism (6) comprises a steering frame (601), a double-shaft motor (602), a worm (603) and a worm gear (604), the worm gear (604) is fixedly installed in one end of the steering frame (601), the worm (603) is fixedly installed at both ends of the double-shaft motor (602), the worm (603) is embedded in the worm gear (604), and the double-shaft motor (602) is fixedly installed in the control mechanism (101).

7. The imaging technology-based pineapple harvesting and transporting integrated robot according to claim 1, characterized in that: The first freedom degree of the mechanical arm (4) is a lifting freedom degree, which is executed by a hydraulic telescopic rod (404); the second freedom degree is a horizontal freedom degree, which is executed by a translation mechanism (402); The third freedom degree is an angle freedom degree, which is executed by an angle adjusting mechanism (405).

8. The imaging technology-based pineapple harvesting and transporting integrated robot according to claim 2, characterized in that: The control mechanism (101) is internally provided with a visual recognition system and a picking navigation system, the visual recognition system performs visual recognition and positioning through a trained model.

9. The imaging-based pineapple harvesting and transporting integrated robot according to any one of claims 1-8, characterized in that: The camera (506) performs visual recognition to determine a pineapple cutting point, the mechanical arm (4) drives the end effector (5) to reach the cutting point, the clamping claw (504) clamps the pineapple and cuts the pineapple connecting handle, and the translation mechanism (402) transports the pineapple into the pineapple collecting frame (302) after cutting.

10. The imaging technology-based pineapple harvesting and transporting integrated robot according to claim 9, characterized in that: The visual recognition is realized through the following steps: (1) a large number of pineapple and pineapple crown images are stored in advance, a pineapple image library is constructed, and deep learning is performed; (2) whether the pineapple fruit is shielded by the stem and leaf is judged, the pineapple point cloud information is acquired through the camera (506), and whether the pineapple fruit is shielded by the stem and leaf is judged; (3) the pineapple bottom coordinate and the pineapple crown coordinate are calculated, the pineapple fruit shielded by the stem and leaf is reconstructed to a three-dimensional surface through a 3D reconstruction unit according to the acquired point cloud information, and computer vision and geometric modeling technology are used to realize; (4) whether the pineapple is inclined is judged, the shortest distance between the pineapple bottom coordinate and the pineapple crown coordinate is calculated through the Floyd algorithm, then the center of the shortest distance is calculated, the center point of the shortest distance is the centroid, the minimum distance from the pineapple bottom to the centroid is calculated, if the inclination angle of the line is less than 20°, the end effector (5) reaches the cutting point for clamping and cutting; if the inclination angle is greater than 70°, the end effector (5) is adjusted in angle and then clamping and cutting are performed; (5) real-time navigation, the real-time tracking data of the camera (506) and the pineapple image data are fused together through the picking navigation system, and the end effector (5) is navigated to the shortest distance of the cutting point in real time.

Images