Portable multi-head picking device and method based on deep learning vision

Abstract

The invention provides a portable multi-head picking device and method based on deep learning vision, and relates to the field of agriculture and forestry picking. Comprising a picking mechanism, a depth camera, a pressure sensor, a calculation module and a control panel. The picking mechanism comprises a six-claw frame, a telescopic rod and a plurality of beating assemblies, and the six-claw frame is connected to one end of the telescopic rod; the multiple beating assemblies are connected to at least part of the claw rods one by one. The depth camera is used for detecting a target area in real time and sending real-time detection data to the calculation module; the pressure sensor is used for detecting whether the beating assembly makes contact with a target fruit or not and sending pressure sensing data to the calculation module. The calculation module is used for generating a control signal in combination with the real-time detection data and the pressure sensing data; the control panel is used for receiving the control signals and controlling the beating parameters of the corresponding beating assemblies according to the control signals. The deep learning vision technology and the multi-head picking mechanism are combined, the picking efficiency can be improved, energy consumption is reduced, and damage to the camellia oleifera trees is reduced.

Description

Technical Field

[0001] The present invention relates to the field of agricultural and forestry picking, and in particular to a portable multi-head picking device based on deep learning vision. Background Art

[0002] Camellia oleifera is one of the four major woody oil plants in the world and a woody oil tree species unique to my country, with a history of cultivation and utilization of more than 2,300 years. Camellia oleifera oil is known as the "Oriental olive oil" because of its unsaturated fatty acid content of more than 90%. It is rich in a variety of trace elements necessary for the human body and has high nutritional and health value. It is the first health care plant oil recommended by the Food and Agriculture Organization of the United Nations. Since mature camellia oleifera fruits are easy to fall off, they need to be harvested in time to ensure the quality of the harvest.

[0003] At present, traditional tea fruit harvesters need to be operated manually to approach the trunk of the tea tree and shake the trunk to make the fruit fall off. However, due to the complex operation of this type of equipment, it is easy to damage the tea tree due to human error, and the picking efficiency and intelligence level are low, which cannot meet the needs of efficient operation in large-scale plantations. In addition, traditional equipment also has obvious shortcomings in energy consumption optimization and equipment endurance. Summary of the invention

[0004] The object of the present invention is to provide a convenient multi-head picking device and method based on deep learning vision, which can improve picking efficiency, reduce energy consumption and reduce damage to tea oil trees.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: A portable multi-head picking device based on deep learning vision, comprising a picking mechanism, a depth camera, a pressure sensor, a computing module and a control board; The picking mechanism comprises a six-claw frame, a telescopic rod and a plurality of groups of beating components, wherein the six-claw frame is connected to one end of the telescopic rod, and the six-claw frame has a plurality of claw rods extending away from the telescopic rod; the plurality of groups of beating components are respectively connected to at least part of the claw rods; The depth camera is installed at the center of the plurality of claw rods and connected to the computing module, and the depth camera is used to detect the target area in real time and send the real-time detection data to the computing module; The pressure sensor is installed on the beating component and connected to the computing module, and the pressure sensor is used to detect whether the beating component contacts the target fruit and send pressure sensing data to the computing module; The computing module is used to analyze the real-time detection data and the pressure sensing data and generate a control signal; The control panel is used to receive a control signal and control the beating parameters of the corresponding beating component according to the control signal.

[0006] Furthermore, in the present invention, the beating assembly includes a motor, a cylindrical cam and two beating rods; the motor is connected to the claw rod through a connecting rod; the cylindrical cam is connected to the output end of the motor, and the side wall of the cylindrical cam is provided with a roller groove; rollers are respectively embedded on opposite sides of the roller groove; the two beating rods are respectively connected to the two rollers in a one-to-one correspondence.

[0007] Furthermore, in the present invention, the beating rod is L-shaped, one end of the cross bar of the beating rod is connected to the roller, the connection part between the cross bar of the beating rod and its vertical bar is hinged to one end of the fixed rod, and the other end of the fixed rod is connected to the side wall of the motor.

[0008] Furthermore, in the present invention, the roller groove is in a periodic wave shape extending along the circumference of the cylindrical cam.

[0009] Furthermore, in the present invention, the picking mechanism also includes an annular fixed disk, and the fixed disk is provided with a plurality of waist holes spaced apart along its circumference; the connecting rod passes through the waist holes and is connected to the claw rod, and the connecting rod is locked to the fixed disk by fasteners; the fixed disk is filled with elastic shock-absorbing material.

[0010] Furthermore, in the present invention, the beating rod is made of flexible material.

[0011] Furthermore, in the present invention, a tray is sleeved on the telescopic rod, and the computing module and the control board are both installed on the tray.

[0012] The present application also provides a picking method based on the above-mentioned picking device, and the method comprises the following steps: The depth camera is used to detect the number of tea fruits in the target area in real time and obtain real-time detection data; The pressure sensor is used to detect whether the corresponding tapping component contacts the target fruit, and obtain pressure sensing data; The real-time detection data and the pressure sensing data are analyzed by a computing module, a control signal is generated and sent to a control board; wherein the computing module has a built-in YOLO11n algorithm; The control board starts the corresponding beating component and controls the beating parameters according to the control signal; wherein the beating parameters are the motor speed and frequency.

[0013] Furthermore, in the present invention, the step of analyzing the real-time detection data and the pressure sensing data by a computing module, generating a control signal and sending it to a control board includes: Comparing the real-time detection data with a first preset threshold to obtain a first comparison result; Comparing the pressure sensing data with a second preset threshold to obtain a second comparison result; When the number of oil-tea camellia fruits reaches a first preset threshold value and the pressure reaches a second preset threshold value, a control signal is generated and sent to the control board.

[0014] Furthermore, in the present invention, the control panel starts the corresponding beating component and controls the beating parameters according to the control signal, including: The control panel controls the rotation speed and frequency of the corresponding beating component through multiple speed stages.

[0015] The present invention has at least the following advantages or beneficial effects: The present invention forms a multi-head picking structure for directly picking fruits by beating through a picking mechanism consisting of a six-claw frame, a telescopic rod and a plurality of groups of beating components. The plurality of groups of beating components can be picked synchronously, which improves the picking efficiency and can reduce the damage to the oil tea tree caused by the vibration and shaking trunk picking method. A depth camera is arranged at the center of the six-claw frame, and a deep learning algorithm is used to identify the number and density of oil tea fruits in real time, and the real-time detection data is sent to a computing module. At the same time, a pressure sensor installed on the beating component is used to detect whether the beating component is in contact with the target fruit, and the detected pressure sensing data is sent to the computing module. The computing module analyzes the real-time detection data and the pressure sensing data, and generates a control signal when it is detected that the number of oil tea fruits reaches a preset threshold and it is detected that the beating component is in contact with the target fruit. The control signal is received by a control panel, and the beating component with pressure data is started according to the control signal, and the beating frequency and rotation speed of the beating component are adjusted according to the detected number and density of oil tea fruits. It realizes accurate identification of the location, size and distribution of tea-oil fruits and accurate picking actions; and only performs picking operations on the target area to avoid damage to surrounding branches and leaves, while reducing energy consumption. Combining deep learning visual technology with a multi-head picking mechanism, it optimizes picking efficiency and intelligence, reduces energy consumption and improves equipment endurance. The device has a compact structure and efficient operation, and is suitable for picking tea-oil fruits and other economic crops, with significant practical value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 A schematic diagram of the overall structure of a portable multi-head picking device based on deep learning vision provided in the application embodiment; Figure 2 A schematic diagram of the structure of multiple flapping components provided for the application embodiment; Figure 3 A schematic diagram of the structure of a single flapping assembly provided for an embodiment of the application; Figure 4 A schematic diagram of the structure of a tray provided in an application embodiment.

[0018] Figure markings: 1-pressure sensor, 2-flapping rod, 3-cylindrical cam, 4-motor, 5-fixed disc, 6-depth camera, 7-connecting rod, 8-six-claw frame, 9-telescopic rod, 10, control board, 11-tray, 12-computing module, 13-telescopic rod adjustment part, 14-fixed rod, 15-hinge, 16-roller, 17-roller groove, 18-motor protection cover, 19-waist hole. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0020] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Example

[0021] Please refer to Figure 1-Figure 4 , which is a schematic diagram of the structure of a portable multi-head picking device based on deep learning vision in an embodiment of the present invention; This embodiment provides a portable multi-head picking device based on deep learning vision, including a picking mechanism, a depth camera 6, a pressure sensor 1, a computing module 12 and a control board 10; The picking mechanism includes a six-claw frame 8, a telescopic rod 9 and a plurality of groups of beating components. The six-claw frame 8 is connected to one end of the telescopic rod 9. The six-claw frame 8 has a plurality of claw rods extending back to the telescopic rod 9. The plurality of groups of beating components are respectively connected to at least part of the claw rods. The depth camera 6 is installed at the center of the plurality of claw rods and connected to the computing module 12. The depth camera 6 is used to detect the target area in real time and send the real-time detection data to the computing module 12; The pressure sensor 1 is installed on the beating assembly and connected to the computing module 12. The pressure sensor 1 is used to detect whether the beating assembly contacts the target fruit and send the pressure sensing data to the computing module 12; The computing module 12 is used to analyze the real-time detection data and pressure sensing data and generate a control signal; The control panel 10 is used to receive a control signal and control the beating parameters of the corresponding beating component according to the control signal.

[0022] Below, a portable multi-head picking device based on deep learning vision according to this exemplary embodiment will be further described.

[0023] In some embodiments of the present application, reference Figure 1 The above-mentioned picking mechanism includes a six-claw frame 8, a telescopic rod 9 and multiple groups of beating components. The six-claw frame 8 is connected to one end of the telescopic rod 9. The telescopic rod 9 can control the length of the device to be adjusted within the range of 0.5 meters to 2 meters. It can be adjusted through the telescopic rod adjusting piece 13 to meet the picking needs of different tree crown heights. The telescopic rod adjusting piece 13 adopts a multi-stage buckle design, which can adjust the length according to the tree crown height to meet the picking needs of low or tall tree layers, and supports fast adjustment and firm locking. The above-mentioned six-claw frame 8 has multiple claw rods extending back to the telescopic rod 9; multiple groups of beating components are respectively connected to at least part of the claw rods one by one to form a multi-head picking structure, which realizes the synchronous picking of multiple target areas, thereby greatly improving the picking efficiency and meeting the distribution needs of different branches.

[0024] The above-mentioned depth camera 6 is the core sensing component of the device, which is installed at the center of multiple claw rods and connected to the computing module 12. The depth camera 6 is used to detect the target area in real time and send the real-time detection data to the computing module 12. At the same time, the pressure sensor 1 is installed on the beating component and connected to the computing module 12. The pressure sensor 1 is used to detect whether the beating component contacts the target fruit and send the pressure sensing data to the computing module 12. The computing module 12 runs the improved deep learning algorithm to quickly analyze and process the real-time detection data and pressure sensing data, generate corresponding control signals, transmit the control signals to the control board 10 via WiFi, and transmit them to the beating component through the control board 10, so as to drive the picking mechanism to perform precise actions. Specifically, when the target fruit is sensed, the corresponding beating component is started, and the beating component is controlled to complete the picking action according to the number and density of the detected oil tea fruits, thereby achieving precise picking, and the beating component that does not sense the fruit does not work, which can greatly reduce energy consumption and avoid damage to the oil tea branches.

[0025] As a preferred implementation method, refer to Figure 2 and Figure 3The above-mentioned beating assembly includes a motor 4, a cylindrical cam 3 and two beating rods 2; the motor 4 is connected to the claw rod through a connecting rod 7, and the connecting rod 7 connects the six-claw frame 8 to the main structure to ensure the stability of the beating assembly during operation. The above-mentioned cylindrical cam 3 is connected to the output end of the motor 4, and the side wall of the cylindrical cam 3 is provided with a roller groove 17; rollers 16 are respectively embedded on opposite sides of the roller groove 17, and the roller groove 17 is used to guide the movement of the roller 16; the two beating rods 2 are respectively connected to the two rollers 16 in a one-to-one correspondence. As the core power component, the cylindrical cam 3 converts the rotational power transmitted by the motor 4 into the periodic swing of the beating rod 2 through the precise fit of the roller 16 embedded in the roller groove 17, and drives the contact to complete the picking action.

[0026] Specifically, the cylindrical cam 3, as the core component of power transmission, drives the rotation of the cylindrical cam 3 through the stable power output of the motor 4. The roller 16 is embedded in the roller groove 17 of the cylindrical cam 3 and can move in the roller groove 17. When the cylindrical cam 3 rotates, the roller 16 performs periodic motion along the trajectory of the roller groove 17, thereby driving the beating rod 2 to complete a precise swinging motion. The beating rod 2 is driven by the roller 16 to ensure the smoothness and efficiency of the picking action. Each beating rod 2 is equipped with a pressure sensor 1 on the inner side, which can sense the presence of the target fruit in real time and accurately control the picking action according to the picking needs, thereby avoiding invalid picking and damage to the fruit trees. The depth camera 6 provides a visual reference for the action of the beating rod 2 by monitoring the target area in real time, further improving the accuracy of picking.

[0027] As a preferred embodiment, the above-mentioned beating bar 2 is L-shaped, one end of the horizontal bar of the beating bar 2 is connected to the roller 16, and the connecting part of the horizontal bar of the beating bar 2 and its vertical bar is hinged to one end of the fixed rod 14 through the hinge 15, and the other end of the fixed rod 14 is connected to the side wall of the motor 4. The beating bar 2 relies on the fixed rods 14 on both sides to provide rotation support during the movement, ensuring that its swing amplitude and angle are always within the design range, which not only meets the picking needs, but also avoids unnecessary damage to the fruits and branches. The above-mentioned beating bar 2 and the fixed rod 14 form a support and swing mechanism, which can ensure the stability and flexibility of the beating bar 2 during the swinging process.

[0028] As a preferred embodiment, the roller groove 17 is in a periodic wave shape extending along the circumference of the cylindrical cam 3, so that the motion trajectory of the roller 16 is a periodic wave trajectory, thereby ensuring that the periodic swing amplitude and frequency of the beating rod 2 are stable.

[0029] As a preferred implementation method, refer to Figure 2The picking mechanism also includes an annular fixed disc 5, which is provided with a plurality of waist holes 19 at intervals along its circumference; the connecting rod 7 passes through the waist holes 19 to connect with the claw rod, and the connecting rod 7 is locked on the fixed disc 5 by fasteners; the fixed disc 5 is filled with high elastic shock-absorbing material. The fixed disc 5 is installed at the bottom of the flapping assembly, and its function is to stabilize the structure of the device and provide overall support for the device. The interior of the fixed disc 5 is filled with shock-absorbing material, and the vibration generated during the picking process is effectively absorbed to ensure that the device remains stable and durable under high-frequency operation, and prevent the loosening of components of the device during high-frequency operation, thereby further improving the reliability of the operation of the device. In addition, by opening a plurality of waist holes 19 on the fixed disc 5, the connecting rod 7 passes through the waist holes 19 to connect with the claw rod of the six-claw frame 8 and is fastened by fasteners such as bolts, so as to support the disassembly and spacing adjustment of the flapping assembly to adapt to the distribution of branches of different densities and diameters. Through the modular design of the six-claw frame 8, the fixed disc 5 and the flapping assembly, the picking range is wider and the efficiency is higher.

[0030] As a preferred implementation method, refer to Figure 1 and Figure 4 The telescopic rod 9 is provided with a tray 11, and the computing module 12 and the control board 10 are both installed on the tray 11. The tray 11 provides a stable physical support for the computing module 12 and related circuit components, ensuring that the equipment runs smoothly in a complex picking environment, making the overall design compact and efficient.

[0031] As a preferred embodiment, the motor 4 is provided with a motor protection cover 18, which protects the internal motor 4 and drive components, and prevents the external environment such as dust or moisture from affecting key components, thereby extending the service life of the equipment.

[0032] As a preferred embodiment, the beating rod 2 is made of a flexible material, and the flexible material may be silicone, polyurethane or other highly elastic materials to reduce damage to the fruit during the picking process.

[0033] As a preferred implementation, the above-mentioned depth camera 6 is OAK-D-Pro, the computing module 12 is NVIDIAJetson Orin NX 16GB, and the control board 10 is ESP32 single-chip microcomputer and stepper motor 4 driver. NVIDIA Jetson OrinNX 16GB is used as the computing core to run the improved deep learning algorithm, and work with the depth camera 6 to identify the position, quantity and distribution of oil tea fruit in real time, and efficiently analyze and process the collected data to generate accurate control signals. The generated control signal is transmitted to the drive system through the ESP32 single-chip microcomputer and the motor 4 driver, and the frequency and speed of the motor 4 are accurately adjusted to control the movement amplitude and rhythm of the beating rod 2, so as to achieve efficient picking of the target fruit. The close cooperation between the various components enables the system to achieve seamless connection from target detection to picking action, greatly improving the automation and accuracy of picking.

[0034] The embodiment of the present application also provides a picking method based on the above-mentioned picking device, comprising the following steps: S110, detecting the number of tea-oil fruits in the target area in real time through the depth camera 6 to obtain real-time detection data.

[0035] S120, detecting whether the corresponding tapping component contacts the target fruit through the pressure sensor 1, and obtaining pressure sensing data.

[0036] S130, analyzing the real-time detection data and pressure sensing data through the computing module 12, generating a control signal and sending it to the control board 10; wherein the computing module 12 has a built-in YOLO11n algorithm.

[0037] S140, the control board 10 starts the corresponding beating component according to the control signal and controls the beating parameters; wherein the beating parameters are the speed and frequency of the motor 4.

[0038] As a preferred implementation, the above step S130, analyzing the real-time detection data and the pressure sensing data by the computing module 12, generating a control signal and sending it to the control board 10, specifically includes: The real-time detection data is compared with a first preset threshold to obtain a first comparison result; the pressure sensing data is compared with a second preset threshold to obtain a second comparison result; the above comparison result is specifically, when the number of tea fruits reaches the first preset threshold, such as the first preset threshold is N, then when the detected number of tea fruits is greater than or equal to N, and the pressure reaches the second preset threshold, such as the second preset threshold is M, when the detected pressure data is greater than or equal to M, that is, a large number of tea fruits are detected and a pressure signal is sensed, a control signal is generated and sent to the control board 10, and the control board 10 drives the motor 4 of the corresponding beating component to operate according to the control signal, and only swings and beats the beating rod 2 with pressure sensing, and automatically adjusts the speed of the motor 4 according to the detected density of the tea fruits to realize power conversion, optimize energy consumption and endurance, and the stepper motor 4 of the beating rod 2 that does not sense pressure remains stationary, thereby achieving precise picking.

[0039] As a preferred implementation method, in the above-mentioned step S140, the control board 10 starts the corresponding beating component and controls the beating parameters according to the control signal, specifically including: the motor 4 control board 10 controls the frequency and rotation speed of the oil tea fruit beating component through multi-stage speed, that is, the frequency and rotation speed of the corresponding segments are adjusted for oil tea fruits of different densities to adapt to the picking requirements of different densities and quantities.

[0040] The present invention combines deep learning vision technology with a multi-head picking mechanism to optimize picking efficiency and intelligence, reduce energy consumption and improve equipment endurance. The device has a compact structure and efficient operation, is suitable for picking oil-tea fruits and other economic crops, and has significant practical value and promotion prospects.

[0041] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A portable multi-head picking device based on deep learning vision, characterized in that: It includes a picking mechanism, a depth camera, a pressure sensor, a computing module and a control board; The picking mechanism comprises a six-claw frame, a telescopic rod and a plurality of groups of beating components, wherein the six-claw frame is connected to one end of the telescopic rod, and the six-claw frame has a plurality of claw rods extending away from the telescopic rod; the plurality of groups of beating components are respectively connected to at least part of the claw rods; The depth camera is installed at the center of the plurality of claw rods and connected to the computing module, and the depth camera is used to detect the target area in real time and send the real-time detection data to the computing module; The pressure sensor is installed on the beating component and connected to the computing module, and the pressure sensor is used to detect whether the beating component contacts the target fruit and send pressure sensing data to the computing module; The computing module is used to analyze the real-time detection data and the pressure sensing data and generate a control signal; The control panel is used to receive a control signal and control the beating parameters of the corresponding beating component according to the control signal.

2. The portable multi-head picking device based on deep learning vision according to claim 1 is characterized in that: The beating assembly includes a motor, a cylindrical cam and two beating rods; the motor is connected to the claw rod through a connecting rod; the cylindrical cam is connected to the output end of the motor, and the side wall of the cylindrical cam is provided with a roller groove; rollers are respectively embedded on opposite sides of the roller groove; the two beating rods are respectively connected to the two rollers in a one-to-one correspondence.

3. The portable multi-head picking device based on deep learning vision according to claim 2 is characterized in that: The beating rod is L-shaped, one end of the cross bar of the beating rod is connected to the roller, the connecting part of the cross bar of the beating rod and its vertical rod is hinged to one end of the fixed rod, and the other end of the fixed rod is connected to the side wall of the motor.

4. The portable multi-head picking device based on deep learning vision according to claim 2 is characterized in that: The roller groove is in a periodic wave shape extending along the circumference of the cylindrical cam.

5. The portable multi-head picking device based on deep learning vision according to claim 2 is characterized in that: The picking mechanism also includes an annular fixed disk, which is provided with a plurality of waist holes spaced apart along its circumference; the connecting rod passes through the waist holes and is connected to the claw rod, and the connecting rod is locked on the fixed disk by a fastener; the fixed disk is filled with elastic shock-absorbing material.

6. The portable multi-head picking device based on deep learning vision according to claim 2 is characterized in that: The beating rod is made of flexible material.

7. The portable multi-head picking device based on deep learning vision according to claim 1 is characterized in that: A tray is sleeved on the telescopic rod, and the computing module and the control panel are both installed on the tray.

8. A picking method based on the picking device according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: The depth camera is used to detect the number of tea fruits in the target area in real time and obtain real-time detection data; The pressure sensor is used to detect whether the corresponding tapping component contacts the target fruit, and obtain pressure sensing data; The real-time detection data and the pressure sensing data are analyzed by a computing module, a control signal is generated and sent to a control board; wherein the computing module has a built-in YOLO11n algorithm; The control board starts the corresponding beating component and controls the beating parameters according to the control signal; wherein the beating parameters are the motor speed and frequency.

9. The picking method according to claim 8, characterized in that: The step of analyzing the real-time detection data and the pressure sensing data by a computing module, generating a control signal and sending the control signal to a control panel comprises: Comparing the real-time detection data with a first preset threshold to obtain a first comparison result; Comparing the pressure sensing data with a second preset threshold to obtain a second comparison result; When the number of oil-tea camellia fruits reaches a first preset threshold value and the pressure reaches a second preset threshold value, a control signal is generated and sent to the control board.

10. The picking method according to claim 8, characterized in that: The control panel starts the corresponding beating component and controls the beating parameters according to the control signal, including: The control panel controls the rotation speed and frequency of the corresponding beating component through multiple speed stages.

Images