Intelligent Fruit Harvesting Device with Robotic Arm Based on Radar Array and RGB-D Camera Fusion Positioning and Its Control Method

Abstract

This invention discloses a robotic arm-based intelligent fruit-picking device and its control method based on radar array and RGB-D camera fusion positioning. The device includes a robotic arm, a lidar, a collection box, and a fruit 3D imaging device, all fixedly mounted on a navigation trolley. A robotic gripper is mounted on the free end of the robotic arm, and a fruit transfer device is installed between the gripper and the collection box. The gripper, robotic arm, lidar, navigation trolley, and fruit 3D imaging device are all electrically connected to a controller. The controller uses the lidar's positioning information to move the navigation trolley to the target picking area and controls the fruit 3D imaging device to perceive multimodal information, acquire fruit maturity and location information, and then adaptively plans the fruit-picking task sequence and the robotic arm's movement path. The controller then controls the gripper to pick the fruit and transports it to the collection box via the fruit transfer device. This invention effectively improves fruit positioning accuracy and fruit-picking capability, and has broad application prospects.

Description

Technical Field

[0001] This invention relates to agricultural intelligent machinery technology, and in particular to a robotic arm intelligent fruit picking device and its control method based on radar array and RGB-D camera fusion positioning. Background Technology

[0002] Traditional fruit picking is done manually, which has problems such as large workload and low picking efficiency. Using machinery to replace manual labor is an effective way to improve picking efficiency. However, the accuracy and robustness of fruit positioning in different environments in existing automatic fruit picking technologies need to be improved. In addition, they all locate and identify surface fruits and cannot detect fruits hidden by leaves or other objects. They cannot perform multi-layer fruit positioning, which affects the fruit picking effect. Summary of the Invention

[0003] Purpose of the invention: One purpose of this invention is to provide a robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning.

[0004] Another objective of this invention is to provide a control method for a robotic arm-based intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning, which can effectively solve the problems of low efficiency in manual fruit picking and poor performance of existing fruit positioning technologies, thereby achieving intelligent and efficient fruit picking.

[0005] Technical Solution: The present invention relates to a robotic arm intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning, comprising a robotic claw, a robotic arm, a lidar, a navigation trolley, a collection box, a fruit transfer device, a fruit 3D imaging device, and a controller. The robotic arm, lidar, collection box, and fruit 3D imaging device are all fixedly mounted on the navigation trolley. The fruit 3D imaging device includes a MIMO radar array and an RGB-D camera. The robotic claw is mounted on the free end of the robotic arm. The fruit transfer device is located directly below the robotic claw, with one end fixedly connected to the free end of the robotic arm and the other end connected to the collection box. The controller is installed inside the navigation trolley and is electrically connected to the robotic claw, robotic arm, lidar, navigation trolley, and fruit 3D imaging device.

[0006] The controller uses the positioning information from the lidar to guide the navigation trolley to the target area to be picked, and controls the MIMO radar array and RGB-D camera of the fruit 3D imaging device to perceive multimodal information, obtain the fruit ripeness and location information, and then adaptively plans the fruit picking task sequence and the robotic arm movement path based on the fruit ripeness and location information, controls the robotic claw to pick the fruit, and transports it to the collection box through the fruit conveying device.

[0007] Preferably, the fruit 3D imaging device further includes an internal gear, an external gear, a parallelogram hinge mechanism, and a drive motor; the internal gear is fixedly mounted on the internal gear shaft, and the external gear is fixedly mounted on the external gear shaft, both of which are supported on the side of the navigation trolley by bearings; the drive motor drives the external gear shaft and the external gear to rotate through a coupling, and the internal gear meshes with the external gear for transmission; the frame of the parallelogram hinge mechanism is fixed on the navigation trolley, and its connecting rod is fixed on the rim of the internal gear and rotates with the internal gear; the MIMO radar array and the RGB-D camera are fixed to the parallelogram hinge mechanism and move with the connecting rod, thereby ensuring that the MIMO radar array and the RGB-D camera are always horizontally positioned relative to the upper surface of the navigation trolley and move with the rotation of the external gear.

[0008] Preferably, the MIMO radar array is a planar rectangular array arranged at equal intervals, using a low-frequency ultra-wideband pulse radar, with the radar spacing within half a wavelength.

[0009] Preferably, the fruit 3D imaging device is fixedly installed on the side of the navigation mobile vehicle, the robotic arm and lidar are fixedly installed on the top of the navigation mobile vehicle, and the robotic claw is installed on the six-dimensional force sensor at the free end of the robotic arm.

[0010] Preferably, the fruit conveying device is a flexible conduit.

[0011] Based on the same inventive concept, the control method of the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning of the present invention includes the following steps:

[0012] S1. The navigation mobile trolley uses LiDAR to locate and drive the robotic arm to move to the target area to be harvested;

[0013] S2. The fruit 3D imaging device is started, and the drive motor drives the internal gear to rotate, which in turn drives the external gear, parallelogram hinge mechanism, MIMO radar array and RGB-D camera to move. The MIMO radar array and RGB-D camera are used to sense the multimodal information of the fruit in the picking area at multiple locations. The multimodal information obtained at multiple locations is stitched together to obtain the fruit ripeness and location information.

[0014] The multimodal information includes a stitched image of a synthesized 3D image, a depth image, and an RGB image;

[0015] S3. The controller adaptively plans the fruit picking task sequence and the robotic arm movement path based on the spatial location information of each mature fruit in the ground coordinate system.

[0016] S4. After the robotic arm moves to the picking position, the controller controls the robotic claw to pick the fruit and send it into the collection box through the fruit conveying device.

[0017] Furthermore, step S2 includes the following steps:

[0018] S21 and MIMO radar arrays penetrate the leaves to obtain radar array signals containing information about the fruit's location, including information about the surface and leaf obstruction, and obtain a synthetic three-dimensional image;

[0019] The S22 and RGB-D cameras capture depth images and RGB images of visible fruit on the surface, and then stitch together multiple sets of depth images and multiple sets of RGB images.

[0020] S23. Based on the color difference of the fruit before and after ripening, the controller uses the stitched RGB image of the fruit as input and uses a support vector machine classification model to determine whether the fruit in the RGB image is ripe.

[0021] S24. Based on the stitched composite 3D image, depth image, and RGB image, obtain the spatial coordinates of each ripe fruit in the ground coordinate system using an end-to-end convolutional neural network model.

[0022] Furthermore, step S4 specifically involves:

[0023] When the robotic gripper is in the open state, it moves to the target position and obtains the force on the gripper through the six-dimensional force sensor at the free end of the robotic arm. It determines whether the appropriate gripping force is used to grasp the fruit. The controller controls the robotic gripper to close until it grasps the fruit. The wrist joint of the robotic gripper rotates to cut the fruit stem, pick the fruit, and then controls the robotic gripper to open. The fruit falls into the fruit conveying device and enters the collection box.

[0024] Based on the same inventive concept, the present invention provides an electronic device comprising a memory and a processor, wherein:

[0025] Memory is used to store computer programs that can run on a processor;

[0026] The processor is configured to execute, while running the computer program, the steps of the control method described above for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning.

[0027] Based on the same inventive concept, the present invention provides a storage medium storing a computer program, which, when executed by at least one processor, implements the steps of the control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning as described above.

[0028] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows:

[0029] This invention utilizes the characteristics of gear transmission and parallelogram hinge mechanism to achieve continuous and stable movement of the MIMO radar array and RGB-D camera through a single motor drive, greatly expanding the imaging range. Considering that fruit positioning is affected by factors such as light, weather, and leaf shading during fruit picking in open-air environments, the device uses a deep fusion and stitching of the synthesized 3D image, depth image, and RGB image to perform 3D positioning of fruits to be picked in various open-air environments with different planting methods on both sides of the road. It autonomously plans the picking task sequence and the movement path of the robotic arm for efficient fruit picking, achieving high-precision and highly robust intelligent fruit picking. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the device structure of the present invention;

[0031] Figure 2 This is a flowchart of the method of the present invention;

[0032] In the diagram: 1. Mechanical claw, 2. Mechanical arm, 3. LiDAR, 4. Navigation mobile vehicle, 5. Collection box, 6. Flexible conduit, 7. Fruit 3D imaging device, 7-1. Internal gear, 7-2. External gear, 7-3. Parallelogram hinge mechanism, 7-4. MIMO radar array, 7-5. RGB-D camera. Detailed Implementation

[0033] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0034] Example 1

[0035] like Figure 1 As shown, the intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning includes a robotic claw 1, a robotic arm 2, a lidar 3, a navigation mobile vehicle 4, a collection box 5, a flexible conduit 6, a fruit 3D imaging device 7, and a controller. The controller is installed inside the navigation mobile vehicle 4 and is used to control the robotic claw 1, robotic arm 2, lidar 3, navigation mobile vehicle 4, and fruit 3D imaging device 7. The robotic claw 1 is mounted on a six-dimensional force sensor at the end of the robotic arm 2. One end of the flexible conduit 6 is connected to the collection box 5, and the other end is connected to the robotic arm 2 through a circular bracket, and is located directly below the robotic claw 1. The bottom of the robotic arm 2 and lidar 3 are fixedly mounted on the top of the navigation mobile vehicle 4 for vehicle positioning. The fruit 3D imaging device 7 is fixedly mounted on the side of the navigation mobile vehicle 4 to sense multimodal information, and after image stitching, obtain the fruit ripeness and location information.

[0036] The fruit 3D imaging device 7 includes an internal gear 7-1, an external gear 7-2, a parallelogram hinge mechanism 7-3, a multiple-input multiple-output (MIMO) radar array 7-4, an RGB-D camera 7-5, and a drive motor. The internal gear 7-1 is fixedly mounted on the internal gear shaft, and the external gear 7-2 is fixedly mounted on the external gear shaft. Both the internal and external gear shafts are supported on the side of the navigation mobile vehicle 4 by bearings. The drive motor drives the external gear shaft and the external gear 7-2 to rotate through a coupling, and the internal gear 7-1 meshes with the external gear 7-2 for transmission. The frame of the parallelogram hinge mechanism 7-3 is fixed on the navigation mobile trolley 4, and its connecting rod is fixed on the rim of the internal gear 7-1, rotating with the internal gear 7-1. The MIMO radar array 7-4 and the RGB-D camera 7-5 are fixed on the parallelogram hinge mechanism 7-3, and move with the connecting rod, so that the MIMO radar array 7-4 and the RGB-D camera 7-5 are always horizontally placed relative to the upper surface of the navigation mobile trolley 4, and move with the rotation of the external gear 7-2, realizing large-range, high-precision side fruit imaging.

[0037] MIMO radar arrays are planar rectangular arrays arranged in 7-4 equally spaced groups. They use low-frequency ultra-wideband pulse radar, which has the ability to penetrate thin obstacles such as blades and has high accuracy. The radar spacing is within half a wavelength.

[0038] Example 2

[0039] like Figure 2 As shown, the control method of the intelligent fruit-picking device with robotic arm based on radar array and RGB-D camera fusion positioning according to the present invention includes the following steps:

[0040] S1. The navigation mobile car 4 is positioned by the lidar 3, which drives the robotic arm 2 to move a certain distance to the target area to be picked; the lidar 3 collects the point cloud signal of the environment around the navigation mobile car 4 and transmits it to the controller, matches the point cloud signal with the global point cloud map, and uses the iterative nearest neighbor (ICP) method to realize the positioning of the navigation mobile car 4, and the navigation mobile car 4 drives the robotic arm 2 to move a certain distance to the target area to be picked;

[0041] S2. The fruit 3D imaging device 7 is activated, and the drive motor drives the internal gear 7-2 to rotate, which in turn drives the external gear 7-1, the parallelogram hinge mechanism 7-3, the MIMO radar array 7-4 and the RGB-D camera 7-5 to move. The MIMO radar array 7-4 and the RGB-D camera 7-5 are used to sense the multimodal information of the fruit in the area to be picked at multiple locations. The multimodal information obtained at multiple locations is stitched together to obtain the fruit ripeness and location information.

[0042] The multimodal information includes a stitched image of a synthesized 3D image, a depth image, and an RGB image. Step S2 includes the following steps:

[0043] S21 and MIMO radar array 7-4 penetrate the leaves to obtain radar array signals containing the location information of the fruit, including the surface and leaf occlusion, and obtain a synthetic three-dimensional image;

[0044] The S22 and RGB-D cameras 7-5 capture depth images and RGB images of visible fruit on the surface, and then stitch together multiple sets of depth images and multiple sets of RGB images.

[0045] S23. Based on the color difference of fruit before and after ripening, use the stitched RGB image of the fruit as input and use the support vector machine (SVM) classification model to determine whether the fruit is ripe.

[0046] S24. Based on an end-to-end convolutional neural network (CNN) model, the model takes the stitched synthetic 3D image, depth image, and RGB image as input and the spatial coordinates of each ripe fruit in the ground coordinate system as output.

[0047] S3. Based on the spatial coordinates of each mature fruit in the ground coordinate system, adaptively plan the fruit picking task sequence and the movement path of the robotic arm.

[0048] Based on the distance between the fruit and the robotic gripper 1, the fruit picking order from near to far is set, and the extended random tree (RRT) method is used to adaptively plan the robotic arm's motion path;

[0049] S4. Control the mechanical claw 1 to pick the fruit and place it in the collection box 5;

[0050] When the mechanical claw 1 is in the open state, after moving to the target position, the force on the mechanical claw 1 is obtained through the six-dimensional force sensor to determine whether the appropriate gripping force is used to grasp the fruit. The controller controls the mechanical claw 1 to close until it grasps the fruit. The wrist joint of the mechanical claw 1 is rotated to cut the fruit stem and pick the fruit. The mechanical claw 1 is then controlled to open, and the fruit falls into the flexible conduit 6 and enters the collection box 5.

[0051] Example 3

[0052] Based on the same inventive concept, the present invention provides an electronic device comprising a memory and a processor, wherein:

[0053] Memory is used to store computer programs that can run on a processor;

[0054] The processor is configured to execute, while running the computer program, the steps of the control method described above for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning.

[0055] The memory may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. The device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the memory may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). A program / utility having a set (at least one) of program modules may be stored in, for example, memory. Such program modules include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules typically perform the functions and / or methods described in the embodiments of the present invention.

[0056] The processor executes various functional applications and data processing by running programs stored in memory, such as the method provided in Embodiment 2 of the present invention.

[0057] Example 4

[0058] Based on the same inventive concept, the present invention provides a storage medium storing a computer program, which, when executed by at least one processor, implements the steps of the control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning as described above.

[0059] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0060] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0061] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0062] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the above-described method operations, but can also perform related operations in the methods provided in any embodiment of the present invention.

[0064] Example 5

[0065] This invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the control method of the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning described in Embodiment 2.

[0066] The system embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0067] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, including read-only memory (ROM), random access memory (RAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-Erasable Programmable Read-Only Memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0068] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of this invention. The embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A robotic arm-based intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning, characterized in that, The system includes a robotic gripper (1), a robotic arm (2), a lidar (3), a navigation mobile vehicle (4), a collection box (5), a fruit transfer device (6), a fruit 3D imaging device (7), and a controller. The robotic arm (2), lidar (3), collection box (5), and fruit 3D imaging device (7) are all fixedly mounted on the navigation mobile vehicle (4). The fruit 3D imaging device (7) includes an internal gear (7-1), an external gear (7-2), a parallelogram hinge mechanism (7-3), a MIMO radar array (7-4), an RGB-D camera (7-5), and a drive motor. The internal gear (7-1) is fixedly mounted on the internal gear shaft, and the external gear (7-2) is fixedly mounted on the external gear shaft. Both the internal and external gear shafts are supported on the side of the navigation mobile vehicle (4) by bearings. The drive motor drives the external gear shaft and the external gear (7-2) to rotate through a coupling. The internal gear (7-1) meshes with the external gear (7-2) for transmission. The parallelogram hinge mechanism... The frame of the chain mechanism (7-3) is fixed on the navigation mobile trolley (4), and its connecting rod is fixed on the rim of the internal gear (7-1) and rotates with the internal gear (7-1); the MIMO radar array (7-4) and the RGB-D camera (7-5) are fixed on the parallelogram hinge mechanism (7-3), and move with the connecting rod, so that the MIMO radar array (7-4) and the RGB-D camera (7-5) are always horizontally placed relative to the upper surface of the navigation mobile trolley (4) and move with the rotation of the external gear (7-2); the mechanical claw (1) is installed at the free end of the mechanical arm (2), and the fruit conveying device (6) is located directly below the mechanical claw (1), one end of which is fixedly connected to the free end of the mechanical arm (2), and the other end is connected to the collection box (5); the controller is installed inside the navigation mobile trolley (4) and is electrically connected to the mechanical claw (1), the mechanical arm (2), the lidar (3), the navigation mobile trolley (4), and the fruit three-dimensional imaging device (7); The controller controls the navigation mobile car (4) to move to the target area to be picked according to the positioning information of the lidar (3), and controls the MIMO radar array (7-4) and RGB-D camera (7-5) of the fruit three-dimensional imaging device (7) to perceive multimodal information, obtain the fruit maturity and location information, and then adaptively plan the fruit picking task sequence and the robotic arm movement path according to the fruit maturity and location information, control the robotic claw to pick the fruit, and transport it to the collection box (5) through the fruit transfer device (6).

2. The robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning as described in claim 1, characterized in that, The MIMO radar array (7-4) is a planar rectangular array with equal intervals, using low-frequency ultra-wideband pulse radar, and the radar spacing is within half a wavelength.

3. The robotic arm intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning as described in claim 1, characterized in that, The fruit 3D imaging device (7) is fixedly installed on the side of the navigation mobile car (4), the robotic arm (2) and the lidar (3) are fixedly installed on the top of the navigation mobile car (4), and the robotic claw (1) is installed on the six-dimensional force sensor at the free end of the robotic arm (2).

4. The robotic arm intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning as described in claim 1, characterized in that, The fruit conveying device (6) is a flexible conduit.

5. A control method for a robotic arm intelligent fruit-picking device based on radar array and RGB-D camera fusion positioning as described in any one of claims 1 to 4, characterized in that, Includes the following steps: S1. The navigation mobile vehicle (4) is positioned by the lidar (3) and drives the robotic arm (2) to move to the target area to be picked; S2. The fruit three-dimensional imaging device (7) is started, and the drive motor drives the internal gear (7-1) to rotate, which in turn drives the external gear (7-2), the parallelogram hinge mechanism (7-3), the MIMO radar array (7-4) and the RGB-D camera (7-5) to move. The MIMO radar array (7-4) and the RGB-D camera (7-5) are used to perceive the multimodal information of the fruit in the picking area at multiple locations. The multimodal information obtained at multiple locations is stitched together to obtain the fruit ripeness and location information. The multimodal information includes a stitched image of a synthesized 3D image, a depth image, and an RGB image; S3. The controller adaptively plans the fruit picking task sequence and the robotic arm movement path based on the spatial location information of each mature fruit in the ground coordinate system. S4. After the robotic arm moves to the picking position, the controller controls the robotic claw to pick the fruit and send it into the collection box (5) through the fruit conveying device.

6. The control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning according to claim 5, characterized in that, Step S2 includes the following steps: S21, MIMO radar array (7-4) penetrates the leaf to obtain radar array signals containing the location information of the fruit on the surface and under the cover of the leaf, and obtains a synthetic three-dimensional image; The S22 and RGB-D cameras (7-5) capture depth images and RGB images of visible fruit on the surface, and then stitch together multiple sets of depth images and multiple sets of RGB images. S23. Based on the color difference of the fruit before and after ripening, the controller uses the stitched RGB image of the fruit as input and uses a support vector machine classification model to determine whether the fruit in the RGB image is ripe. S24. Based on the stitched composite 3D image, depth image, and RGB image, obtain the spatial coordinates of each ripe fruit in the ground coordinate system using an end-to-end convolutional neural network model.

7. The control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning according to claim 5, characterized in that, Step S4 is as follows: When the mechanical claw (1) is in the open state, after moving to the target position, the force of the mechanical claw (1) is obtained by the six-dimensional force sensor at the free end of the mechanical arm. It is determined whether the appropriate gripping force is used to grab the fruit. The controller controls the mechanical claw (1) to close until it grabs the fruit. The wrist joint of the mechanical claw (1) is rotated to cut the fruit stem and pick the fruit. The mechanical claw (1) is then controlled to open, and the fruit falls into the fruit conveying device (6) and enters the collection box (5).

8. An electronic device, characterized in that, Includes memory and processor, wherein: Memory is used to store computer programs that can run on a processor; A processor, configured to, while running the computer program, execute the steps of the control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning as described in any one of claims 5-7.

9. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by at least one processor, implements the steps of the control method for the robotic arm intelligent fruit picking device based on radar array and RGB-D camera fusion positioning as described in any one of claims 5-7.

Images