Fruit picking method, picking device and picking robot

Abstract

The embodiment of the invention relates to the technical field of fruit picking, and discloses a fruit picking method, a picking device and a picking robotic.The method comprises the steps that hidden fruits hidden in a to-be-picked area and the hidden positions of the hidden fruits are determined, and a picking mechanical arm is driven to drive the hidden fruits to be picked; the air nozzle assembly is driven to directly face the hidden position to blow air so as to expose the hidden position, the picking mechanical arm is driven to move towards the hidden position until the clamping mechanism reaches the hidden position, the clamping mechanism is driven to clamp the hidden fruits, the air supply device is controlled to stop supplying air, and the clamping mechanism is driven to be recycled. And picking the hidden fruits. By means of the mode, the fruit sheltered by the leaves can be effectively exposed, and comprehensiveness and thoroughness of picking operation are guaranteed.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of fruit picking, in particular to a fruit picking method, a picking device and a picking robot. BACKGROUND

[0002] With the acceleration of urbanization process, the agricultural population continues to flow out, and every time the picking season comes, fruit farmers will face seasonal labor shortage, and the best picking time suitable for picking mature fruits is very short. Once the best picking time is missed, the picked fruits not only have poor appearance, but also have obvious difference in taste, which affects the yield of fruit farmers. In order to solve the problem of seasonal fruit picking of fruit farmers, currently, picking robots appear on the market, the picking robot includes a wheeled chassis, a mechanical arm, a clamping jaw and a picking basket, the picking basket and the mechanical arm are arranged on the wheeled chassis, the clamping jaw is arranged at the end of the mechanical arm, the mechanical arm drives the clamping jaw to clamp the fruit, and then the fruit is placed in the picking basket to realize the picking of the fruit. The picking robot can replace repetitive labor, especially realize 24-hour continuous operation during the fruit ripening period, and effectively make up for the shortage of manpower.

[0003] In the implementation process of the present application, the inventor found that: at present, the existing fruit picking method usually adopts a double-arm cooperative operation mode, that is, one mechanical arm is responsible for pushing away the shielding leaves, and the other mechanical arm is responsible for clamping the fruit. The existing picking method lacks an effective hidden fruit active search mode, and can usually only handle the exposed fruits that can be directly observed. There is a lack of systematic means for finding hidden fruits shielded by dense leaves. The traditional method mainly relies on passive identification of the visual system. When the fruit is completely shielded by the leaves, the image recognition technology cannot obtain effective fruit feature information, resulting in that the hidden fruit cannot be accurately determined. SUMMARY

[0004] The technical problem solved by the embodiment of the present application is to provide a fruit picking method, which can effectively expose the fruits shielded by leaves through the blowing operation of the air nozzle assembly in the hidden position, ensure the comprehensiveness and thoroughness of the picking operation, avoid the problem of missing hidden fruits in the traditional method, and improve the overall picking efficiency and agricultural production income.

[0005] To solve the above technical problems, one technical scheme adopted by the embodiment of the present application is to provide a fruit picking method applied to a picking robot, the picking robot comprising a movable chassis and a picking mechanical arm, the picking mechanical arm comprising a mechanical arm body, an air nozzle assembly, a gas supply device, a camera device and a clamping mechanism, the mechanical arm body and the gas supply device being arranged on the movable chassis, the clamping mechanism and the air nozzle assembly being arranged at the end of the mechanical arm body, the gas supply device being connected with the air nozzle assembly, the camera device being arranged on the picking mechanical arm, the camera device being used to take an image of a region to be picked, and the method comprising the following steps: determining a hidden fruit hidden in the region to be picked and a hidden position of the hidden fruit, driving the picking mechanical arm to make the air nozzle assembly blow air at the hidden position to expose the hidden position, driving the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position, driving the clamping mechanism to clamp the hidden fruit, and controlling the gas supply device to stop supplying air, and driving the clamping mechanism to recover to pick the hidden fruit.

[0006] Optionally, the step of driving the picking mechanical arm to make the air nozzle assembly blow air at the hidden position to expose the hidden position comprises the following steps: driving the picking mechanical arm to make the air nozzle assembly blow air at the hidden position, controlling the gas supply device to supply air to the air nozzle assembly at a preset wind power level, identifying whether the hidden fruit is exposed after a preset time duration, if the hidden fruit is not exposed, identifying whether the gas supply device reaches a maximum wind power level, if the gas supply device does not reach the maximum wind power level, increasing the wind power level of the gas supply device by one level, and returning to the step of identifying whether the hidden fruit is exposed after a preset time duration until the hidden fruit is identified to be exposed, and if the hidden fruit is exposed, fixing the current wind power level of the gas supply device.

[0007] Optionally, if the wind power level reaches the preset maximum wind power level and the fruit is still not exposed, sending a picking failure alarm information to an external device, the picking failure alarm information comprising the hidden position of the hidden fruit.

[0008] Optionally, if the hidden fruit is identified to be exposed after the gas supply device supplies air to the air nozzle assembly at a preset wind power level for a preset time duration, the wind power of the gas supply device is controlled to be decreased step by step until a minimum wind power level capable of exposing the hidden fruit is found.

[0009] Optionally, before the driving the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position, the method further comprises: blowing air to the air nozzle assembly when the air nozzle assembly is directly opposite the hidden position to expose the hidden position, identifying whether the hidden fruit swings according to the image picked up by the camera; if the hidden fruit swings, determining the swing amplitude of the hidden fruit; determining whether the swing amplitude of the hidden fruit is greater than the maximum opening of the clamping mechanism, if yes, sending a picking failure alarm information to an external device; if the swing amplitude of the hidden fruit is less than the maximum opening of the clamping mechanism, entering the step of driving the picking mechanical arm to move towards the hidden position.

[0010] Optionally, the step of driving the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position further comprises: during the movement of the clamping mechanism to the hidden position, identifying whether the hidden fruit swings according to the image picked up by the camera, identifying whether the hidden fruit swings; if it is identified that the hidden fruit swings, identifying the maximum swing amplitude of the hidden fruit in real time; determining whether the maximum swing amplitude is greater than the maximum opening of the clamping mechanism; when the maximum swing amplitude is greater than the maximum opening of the clamping mechanism, sending a picking failure alarm information to an external device; when the maximum swing amplitude is less than the maximum opening of the clamping mechanism, continuing to drive the clamping mechanism until the clamping mechanism reaches the hidden position.

[0011] Optionally, the picking failure alarm information further comprises an image of the position of the hidden fruit, and the hidden position is marked in the image.

[0012] Optionally, the steps of determining the hidden fruit hidden in the picking area and the hidden position of the hidden fruit further comprise: after all the fruits exposed in the picking area have been picked, controlling the air supply device to supply air to the air nozzle assembly; after the air supply device supplies air to the air nozzle assembly, driving the picking mechanical arm to make the air nozzle assembly traverse the picking area in sequence; during the traversal of the air nozzle assembly in the picking area in sequence, identifying whether there is a hidden fruit according to the image picked up by the camera; if there is, positioning the hidden position of the hidden fruit.

[0013] To solve the above technical problems, another technical solution adopted by the embodiments of the present application is to provide a fruit picking device applied to a picking robot, the picking robot comprising a movable chassis, a picking mechanical arm, an air nozzle assembly, an air supply device, a camera device, and a clamping mechanism, the picking mechanical arm and the air supply device being arranged on the movable chassis, the clamping mechanism and the air nozzle assembly being arranged at the end of the picking mechanical arm, the air supply device being connected with the air nozzle assembly, the camera device being arranged on the picking mechanical arm, the camera device being used to take images of a region to be picked, the fruit picking device comprising: a determination module configured to determine a hidden fruit hidden in the region to be picked and a hidden position of the hidden fruit; a first driving module configured to drive the picking mechanical arm to blow air at the hidden position by the air nozzle assembly to expose the hidden position, a second driving module configured to drive the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position, a third driving module configured to drive the clamping mechanism to clamp the hidden fruit and control the air supply device to stop supplying air, and a recovery module configured to drive the clamping mechanism to recover to pick the hidden fruit.

[0014] Still another technical solution adopted by the embodiments of the present application is to provide a picking robot, comprising: a movable chassis; a picking mechanical arm comprising a mechanical arm body, a clamping mechanism, an air nozzle assembly, and an air supply device, the mechanical arm body and the air supply device being arranged on the movable chassis, the air nozzle assembly and the clamping mechanism being arranged at the end of the mechanical arm body, the air supply device being connected with the air nozzle assembly, the air supply device being configured to supply air to the air nozzle assembly; a camera device arranged on the picking mechanical arm controller and arranged on the movable chassis; and a controller, the controller comprising: at least one processor in communication connection with a memory, the clamping mechanism, the air supply device, and the camera device; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0015] Still another technical solution adopted by the embodiments of the present application is to provide a non-transitory computer readable storage medium, the non-transitory computer readable storage medium storing computer executable instructions, the computer executable instructions being configured to enable a server to perform the method as described above.

[0016] The application provides a fruit picking method applied to a picking robot, the picking robot comprising a movable chassis and a picking mechanical arm, the picking mechanical arm comprising a mechanical arm body, an air nozzle assembly, a gas supply device, a camera device and a clamping mechanism, the mechanical arm body and the gas supply device being arranged on the movable chassis, the clamping mechanism and the air nozzle assembly being arranged at the tail end of the mechanical arm body, the gas supply device being connected with the air nozzle assembly, and the camera device being arranged on the picking mechanical arm and used for shooting images of a region to be picked, and the method is characterized in that the method comprises the following steps: determining a hidden fruit hidden in the region to be picked and a hidden position of the hidden fruit, driving the picking mechanical arm to make the air nozzle assembly blow air directly at the hidden position to expose the hidden position, driving the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position, driving the clamping mechanism to clamp the hidden fruit, and controlling the gas supply device to stop supplying air, and driving the clamping mechanism to recover to pick the hidden fruit. The design of the single mechanical arm and the air nozzle assembly in the application embodiment effectively solves the technical problems of high cost and complex control of the traditional double-arm picking robot. Compared with the traditional scheme that needs to cooperatively control two mechanical arms, only one mechanical arm needs to be cooperated with the air nozzle assembly in the application to realize the search and picking of the hidden fruit. The mechanical arm drives the air nozzle assembly to search in the region to be picked, which can accurately find the fruit hidden by leaves and position the fruit, improves the adaptability of the mechanical arm picking to the complex environment, ensures the comprehensiveness and thoroughness of the picking operation, and effectively avoids the omission of the hidden fruit. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the specific embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the description of the specific embodiments or the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, each element or part is not necessarily drawn according to the actual proportion.

[0018] Figure 1 is a schematic view of the picking mechanical arm in the application embodiment; Figure 2 is a partial enlarged view of A part in Figure 1 Figure 3 is another perspective exploded view of the picking mechanical arm in the application embodiment; Figure 4 is a partial enlarged view of B part in Figure 3 Figure 5 is another partial enlarged view of B part in Figure 3 Figure 6 ​​​is another perspective exploded view of the picking mechanical arm of the embodiment of the present application; Figure 7 is Figure 6 is a partial enlarged view of part C in the figure; Figure 8 is Figure 1 is another partial enlarged view of part A in the figure; Figure 9 is another embodiment of the picking mechanical arm of the embodiment of the present application; Figure 10 is Figure 9 is a partial enlarged view of part D in the figure; Figure 11 is Figure 9 is another partial enlarged view of part D in the figure; Figure 12 is another embodiment of the picking mechanical arm of the embodiment of the present application; Figure 13 is Figure 12 is a partial enlarged view of part E in the figure; Figure 14 is a flow chart of a fruit picking method of the embodiment of the present application; Figure 15 is a further flow chart of step S101 of the embodiment of the present application; Figure 16 is a further flow chart of step S102 of the embodiment of the present application; Figure 17 is a further flow chart of step S103 of the embodiment of the present application; Figure 18 is another flow chart of a fruit picking method of the embodiment of the present application; Figure 19 is a structural block diagram of a fruit picking device of the embodiment of the present application; Figure 20 is a schematic diagram of a picking robot of the embodiment of the present application.

[0019] The specific embodiment is as follows: 100, picking mechanical arm; 10, mechanical arm body; 20, air supply device; 30, clamping mechanism; 31, base; 32, clamping fingers; 33, clamping driving piece; 40, swing mechanism; 41, swing base; 411, first swing part; 401, first through hole; 412, second swing part; 413, third swing part; 405, third through hole; 406, fourth through hole; 414, fourth swing part; 42, first telescopic assembly; 421, first mounting piece; 403, first plug-in part; 422, first telescopic piece; 43, second telescopic assembly; 431, second mounting piece; 404, second plug-in part; 432, second telescopic piece; 44, third telescopic assembly; 441, third mounting piece; 442, third telescopic piece; 407, third plug-in part; 408, fourth plug-in part; 45, fourth telescopic assembly; 451, fourth mounting piece; 452, fourth telescopic piece; 408, fourth plug-in part; 50, air nozzle assembly; 51, air nozzle; 60, elastic conduit; 70, protection assembly; 71, switch driving piece; 72, rotating piece; 73, baffle; 80, picking device; 81, determining module; 82, first driving module; 83, second driving module; 84, third driving module; 85, recycling module; 200, picking robot; 201, movable chassis; 202, loading frame. Specific embodiment

[0020] In order to facilitate the understanding of the present application, the present application will be described in more detail below in conjunction with the drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element or one or more intervening elements can be present therebetween. When an element is described as "connected to" another element, it can be directly connected to the other element or one or more intervening elements can be present therebetween. The terms "upper", "lower", "inner", "outer", "vertical", "horizontal" and the like used in the specification indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0021] Unless otherwise defined, all technical and scientific terms used in the specification have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not used to limit the present application. The term "and / or" used in the specification includes any and all combinations of one or more related listed items.

[0022] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0023] Please refer to Figure 1 and Figure 2 , the picking mechanical arm 100 includes: air supply device 20, mechanical arm body 10, clamping mechanism 30, swing mechanism 40 and air nozzle assembly 50. The clamping mechanism 30 is arranged at the end of the mechanical arm body 10, and the clamping mechanism 30 is used for clamping the object to be clamped. In the present application, the object to be clamped is fruit, so the clamping mechanism 30 undertakes the function of clamping fruit. The swing mechanism 40 is installed on the clamping mechanism 30 and provides motion support for the air nozzle assembly 50. The air nozzle assembly 50 provides directional air blowing to the blade and other obstacles that block the target object through the air flow provided by the air supply device 20, so that the hidden fruit is exposed. The swing mechanism 40 serves as a motion control unit of the air nozzle assembly 50 and can drive the air nozzle assembly 50 to swing within a predetermined angle range. Through the control of the swing mechanism 40, the air nozzle assembly 50 can adjust the direction and angle of air blowing to the obstruction, adapt to the cleaning needs of obstructions at different positions and angles, and the angle range and speed of the swing motion can be adjusted according to the specific working environment.

[0024] The air supply device 20 provides components as an air source to provide stable compressed air for the entire air flow system. The air supply device 20 is provided with structures such as an air pump (not shown in the figure), an air tank (not shown in the figure) and a pressure regulating valve (not shown in the figure). The air pump compresses the external air and stores it in the air tank. The pressure regulating valve adjusts the output air pressure according to the working requirements. The air supply device 20 is connected with the air nozzle assembly 50 through an air pipe to form a complete air path transmission channel. The mechanical arm body 10 includes multiple joints and connecting rods, which are driven by servo motors to move the joints to achieve precise positioning of the end effector in three-dimensional space. The mechanical arm body 10 is made of lightweight aluminum alloy material, which reduces the self-weight while ensuring the structural strength, and improves the motion response speed and positioning accuracy. The clamping mechanism 30 is installed at the end position of the mechanical arm body 10. The clamping mechanism 30 is used for clamping the object to be clamped. In the present application, the object to be clamped is fruit, so the clamping mechanism 30 undertakes the function of clamping fruit. The clamping mechanism 30 is internally provided with a clamping driving device, which controls the opening and closing action of the clamping mechanism 30 to achieve adaptive clamping of fruit of different sizes through coordinated action. Preferably, the surface of the clamping mechanism 30 is covered with soft silicone material to avoid damage to the fruit during clamping. The air nozzle assembly 50 is also arranged at the end of the mechanical arm body 10. The air nozzle assembly 50 is connected with the air supply device 20 to receive the compressed air flow provided by the air supply device 20. The air nozzle assembly 50 is used to blow air to the blades that block the fruit, and the blades are blown open by the air flow to expose the fruit hidden behind the blades.

[0025] In actual picking operation, when the picking manipulator 100 reaches the target operation position, the air nozzle assembly 50 first blows the shielding objects such as leaves and branches shielding the to-be-held objects through the driving of the swing mechanism 40, and the airflow causes the shielding objects to displace, thereby exposing the hidden to-be-held objects. After the shielding objects are removed, the clamping mechanism 30 performs clamping operation on the exposed to-be-held objects, completes the picking action, and through the cooperation of the swing mechanism 40 and the air nozzle assembly 50, the active shielding object processing capability is realized, and the picking manipulator 100 can complete the removal of shielding objects at different angles without large-scale overall position adjustment, thereby improving the operation efficiency. The adjustable blowing angle design of the air nozzle assembly 50 enhances the adaptability of the system to complex environment, and provides a technical basis for precise picking. The integrated design of the air supply device 20 and the air nozzle assembly 50 simplifies the system structure, and reduces the complexity and maintenance cost of the equipment.

[0026] Please refer to Figure 3 and Figure 4 , the swing mechanism 40 includes a swing seat 41, a first telescopic assembly 42 and a second telescopic assembly 43, the swing seat 41 serves as a bearing platform for the air nozzle assembly 50, and provides a stable mounting basis for the air nozzle assembly 50, one end of the first telescopic assembly 42 is rotatably connected with the swing seat 41, and the other end of the one end of the first telescopic assembly 42 is installed on the clamping mechanism 30, one end of the second telescopic assembly 43 is rotatably connected with the swing seat 41, and the other end of the second telescopic assembly 43 is installed on the clamping mechanism 30, the first telescopic assembly 42 and the second telescopic assembly 43 are arranged in a relative arrangement state in space, forming a symmetrical support structure. The air nozzle assembly 50 is arranged on the swing seat 41, and the position and angle are adjusted through the movement of the swing seat 41.

[0027] The first telescopic assembly 42 and the second telescopic assembly 43 have independent extension and retraction control capability, when the first telescopic assembly 42 and the second telescopic assembly 43 perform synchronous extension action, the swing seat 41 extends forward together with the air nozzle assembly 50, increasing the coverage distance of blowing operation. When the first telescopic assembly 42 and the second telescopic assembly 43 perform synchronous retraction action, the swing seat 41 retracts backward together with the air nozzle assembly 50, adjusting the distance from the target area.

[0028] When the first telescopic assembly 42 and the second telescopic assembly 43 perform differential actions, the swing seat 41 generates angular deflection movement. Specifically, when the first telescopic assembly 42 is stretched and the second telescopic assembly 43 remains unchanged or retracts, the swing seat 41 is deflected to the side of the second telescopic assembly 43. Conversely, when the second telescopic assembly 43 is stretched and the first telescopic assembly 42 remains unchanged or retracts, the swing seat 41 is deflected to the side of the first telescopic assembly 42. Through the coordinated control of the two telescopic assemblies, the air nozzle assembly 50 can achieve precise direction adjustment within a preset angle range.

[0029] In the embodiments of the present application, the design of the first telescopic assembly 42 and the second telescopic assembly 43 realizes independent control of the air nozzle assembly 50 in the two movement dimensions of telescoping and swinging. The telescoping characteristics of the first telescopic assembly 42 and the second telescopic assembly 43 adjust the working distance of the air nozzle assembly 50 from the target area, and the swinging control of the first telescopic assembly 42 and the second telescopic assembly 43 adjusts the blowing direction of the air nozzle assembly 50. The two control modes can be executed separately or in combination, providing flexible movement solutions for different operation requirements.

[0030] Further, please continue to refer to Figure 4 , the swing mechanism 40 further comprises a third telescopic assembly 44 and a fourth telescopic assembly 45, one end of the third telescopic assembly 44 is rotatably connected with the swing seat 41, the other end of the one end of the third telescopic assembly 44 is installed on the clamping mechanism 30, one end of the fourth telescopic assembly 45 is rotatably connected with the swing seat 41, the other end of the fourth telescopic assembly 45 is installed on the clamping mechanism 30; The spatial layout of the four telescopic assemblies follows the principle of vertical intersection: the first telescopic assembly 42 and the second telescopic assembly 43 are oppositely arranged along a first direction, the third telescopic assembly 44 and the fourth telescopic assembly 45 are oppositely arranged along a second direction, and the first direction and the second direction are perpendicular, The first, second, third and fourth telescopic assemblies 42, 43, 44 and 45 form two independent control planes. The first and second telescopic assemblies 42 and 43 constitute a first control plane, responsible for controlling the telescopic and swing movement of the air nozzle assembly 50 in a first direction. The third and fourth telescopic assemblies 44 and 45 constitute a second control plane, responsible for controlling the telescopic and swing movement of the air nozzle assembly 50 in a second direction. The two control planes are perpendicular to each other and operate independently. In the embodiment of the present application, through the differential control of the first, second, third and fourth telescopic assemblies 42, 43, 44 and 45, the air nozzle assembly 50 obtains full-range movement capability in a two-dimensional plane. The movement of the first control plane and the movement of the second control plane can be performed simultaneously, realizing the complex movement trajectory of the air nozzle assembly 50 in a composite direction. When the first and second telescopic assemblies 43 perform swing control, the third and fourth telescopic assemblies 45 perform swing control in another direction, and the air nozzle assembly 50 can reach any angular position in the working space.

[0031] Further, please refer to Figure 5 、 Figure 6 and Figure 7 , the swing seat 41 is provided with a first swing part 411 and a second swing part 412. The first swing part 411 is provided with a first through hole 401, which is a connection interface of the first telescopic assembly 42. The second swing part 412 is provided with a second through hole (not shown in the figure), which is a connection interface of the second telescopic assembly 43. The first telescopic assembly 42 comprises a first mounting member 421 and a first telescopic member 422. One end of the first telescopic member 422 is fixed to the clamping mechanism 30, and the other end of the first telescopic member 422 is rotationally connected to the first mounting member 421. The first telescopic member 422 bears the length adjustment function. The first mounting member 421 serves as a connection intermediary and is provided with a first plug-in part 403 at the end part. The first plug-in part 403 has a cylindrical or other suitable geometric shape and can be rotationally inserted into the first through hole 401 to realize rotational connection with the swing seat 41.

[0032] The second telescopic assembly 43 comprises a second mounting member 431 and a second telescopic member 432. One end of the second telescopic member 432 is fixed to the clamping mechanism 30, and the other end of the second telescopic member 432 is rotationally connected to the second mounting member 431. The second mounting member 431 is provided with a second plug-in part 404, which is rotationally inserted into the second through hole (not shown in the figure) to form a movable connection with the swing seat 41. The symmetrical design of the first and second telescopic assemblies 42 and 43 ensures the balance and stability of the movement of the swing seat 41.

[0033] In the embodiment of the present application, the first telescopic assembly 42 and the second telescopic assembly 43 can transmit the length change to the swing base 41, so as to control the position and angle of the air nozzle assembly 50, and the rotation accuracy of each plug-in connection point directly affects the positioning accuracy of the entire swing system, and the high-precision plug-in connection ensures the repeatability and controllability of the swing movement.

[0034] Please refer to the continuation Figure 5 and Figure 7 , the swing base 41 is provided with a third swing part 413 and a fourth swing part 414, the third swing part 413 is provided with a third through hole 405, and the fourth swing part 414 is provided with a fourth through hole 406, the third swing part 413 is provided with the third through hole 405, the fourth swing part 414 is provided with the fourth through hole 406, the distribution positions of the four swing parts on the swing base 41 are coordinated with the spatial layout of the corresponding telescopic assemblies, and a stable four-point connection structure is formed. Appropriate structural spacing is maintained between the swing parts to avoid movement interference between adjacent connection components.

[0035] The third telescopic assembly 44 includes a third mounting piece 441 and a third telescopic piece 442, one end of the third telescopic piece 442 is fixed to the clamping mechanism 30, the other end of the third telescopic piece 442 is rotationally connected with the third mounting piece 441, length adjustment and angle transmission are realized, and the third mounting piece 441 is provided with a third plug-in part 407, the third plug-in part 407 is rotationally arranged in the third through hole 405, and a rotation connection relationship with the swing base 41 is established.

[0036] The fourth telescopic assembly 45 includes a fourth mounting piece 451 and a fourth telescopic piece 452, one end of the fourth telescopic piece 452 is fixed to the clamping mechanism 30, the other end of the fourth telescopic piece 452 is rotationally connected with the fourth mounting piece 451, the fourth mounting piece 451 is provided with a fourth plug-in part 408, the fourth plug-in part 408 is rotationally arranged in the fourth through hole 406, and the third telescopic assembly 44, the fourth telescopic assembly 45, the first telescopic assembly 42 and the second telescopic assembly 43 are vertically crossed in space, forming a complete quadrilateral support frame.

[0037] In the embodiment of the present application, the plug-in connection system of the first telescopic assembly 42, the second telescopic assembly 43, the third telescopic assembly 44 and the fourth telescopic assembly 45 realizes high-precision composite motion control of the air nozzle assembly 50 in two vertical planes. The length adjustment accuracy of each telescopic assembly is transmitted to the swing base 41 through the plug-in connection mode, so as to realize the reaching of the air nozzle assembly 50 at any position in the working space.

[0038] In the embodiment of the present application, the picking mechanical arm 100 further comprises an elastic conduit 60, and the air nozzle assembly 50 is connected with the air supply device 20 through the elastic conduit 60. The elastic conduit 60 is made of a high-molecular elastic material and has excellent flexibility and elastic deformation capability. Preferably, the two ends of the elastic conduit 60 are respectively provided with special connecting joints to realize reliable connection with the air supply device 20 and the air nozzle assembly 50. The connecting joint at the end of the air supply device 20 adopts a standardized quick-connection structure to facilitate installation and maintenance. The connecting joint at the end of the air nozzle assembly 50 adopts a rotary joint or a universal joint design to avoid excessive torsional stress of the conduit during swinging. Preferably, the connecting joint of the elastic conduit 60 is provided with a sealing ring or a sealing gasket to ensure the sealing performance of the air path system. The conduit material is selected to have good pressure resistance so as to be able to withstand the working pressure output by the air supply device 20 without rupture or deformation. The pressure resistance grade of the elastic conduit 60 is designed to have a proper safety margin to improve the reliability and service life of the system.

[0039] In the embodiment of the present application, the swinging mechanism 40 further comprises an elastic net, one end of the elastic net is fixed to the swinging seat 41, and the other end of the elastic net is fixed to the clamping mechanism 30. The elastic net is arranged around the part of the elastic conduit 60 between the clamping mechanism 30 and the swinging seat 41. The elastic net adopts a net structure design and has good flexibility and air permeability. The elastic net is arranged around the part of the elastic conduit 60 between the clamping mechanism 30 and the swinging seat 41 to form a complete protection space.

[0040] In the embodiment of the present application, the number of the swinging mechanisms 40 is multiple. Preferably, in the embodiment of the present application, the number of the swinging mechanisms 40 is three. The three swinging mechanisms 40 are uniformly distributed along the circumference of the clamping mechanism 30, and the angular interval between adjacent swinging mechanisms 40 is 120 degrees. The first swinging mechanism is installed at the front position of the clamping mechanism, the second swinging mechanism is installed at the left front 60-degree position, and the third swinging mechanism is installed at the right front 60-degree position. The installation base of each swinging mechanism is rigidly connected with the clamping mechanism 30 to ensure that the three swinging mechanisms 40 have a unified movement reference. Each swinging mechanism 40 is provided with an independent air nozzle assembly 50 to form three air flow output units. The three air nozzle assemblies 50 have the same model specification and have the same air flow output characteristics and control accuracy. Each air nozzle assembly 50 is connected with the air supply device 20 through an independent elastic conduit 60 to realize independent air flow supply and control. The three air nozzle assemblies 50 can work simultaneously or independently, and are flexibly configured according to the operation requirements.

[0041] In the embodiment of the present application, the three swing mechanisms 40 system provides more precise blowing angle adjustment capability. The combined work of any two swing mechanisms 40 can produce a synthetic air flow effect, and by adjusting the angle and air flow intensity of each swing mechanism 40, the direction and intensity of the synthetic air flow can be accurately controlled. The full combination work mode of the three swing mechanisms 40 can produce a complex air flow distribution pattern, adapt to different shapes and distribution of the barrier removal requirements, and the three swing mechanisms 40 greatly expand the coverage of the air flow operation. The operation fan-shaped area of a single swing mechanism 40 and the operation area of the other two swing mechanisms 20 form an overlapping coverage, eliminating the operation blind area. The three swing mechanisms 40 can comprehensively cover the approximately hemispherical space in front of the clamping mechanism, adapt to complex barrier distribution conditions. The air flow superposition effect in the overlapping area enhances the barrier removal capability and improves the operation efficiency.

[0042] Further, the three swing mechanisms 40 system provides more precise blowing angle adjustment capability. The combined work of any two swing mechanisms 40 can produce a synthetic air flow effect, and by adjusting the angle and air flow intensity of each swing mechanism 40, the direction and intensity of the synthetic air flow can be accurately controlled. The full combination work mode of the three swing mechanisms 40 can produce a complex air flow distribution pattern, adapt to different shapes and distribution of the barrier removal requirements. When one of the swing mechanisms 40 fails, the remaining two swing mechanisms 40 can still maintain basic operation capability, ensuring the continuity of the picking operation. The independent design of the three swing mechanisms 40 makes it more convenient to isolate and maintain the failure, reducing the maintenance cost and downtime of the system.

[0043] In the embodiment of the present application, please refer to Figure 9 , Figure 10 and Figure 11 , in particular, please refer to Figure 11, the air nozzle assembly 50 includes a plurality of air nozzles 51, which are arranged in a ring shape on the circumference of the air nozzle assembly 50, and the ring-shaped arrangement ensures that the air flow can uniformly cover the area around the clamping mechanism 30, effectively dealing with the situation of blade shielding in different directions. The flow channel of the air nozzle 51 is arranged in an inclined manner, specifically, the air outlet direction of the air nozzle 51 is inclined at a certain angle relative to the axis of the air nozzle assembly 50, so that the air flow ejected by each air nozzle 51 forms a conical diffusion state in space. The flow channels of the plurality of air nozzles 51 constitute an outward scattering layout, and the air flow spreads from the center area to the surrounding area, forming an effective blade pushing coverage range. Through the outward scattering layout of the flow channels of the plurality of air nozzles 51, firstly, the diffusion air flow has a much larger coverage area than the single-point straight air flow, and can simultaneously handle blade shielding in multiple directions. Secondly, the scattering air flow gradually decays in the propagation process, avoiding causing damage to the fruits due to excessive impact. Finally, the ring-shaped scattering design is well matched with the clamping mechanism 30, and the air flow passes through the gap between the clamping fingers 32 of the clamping mechanism 30 without interfering with the execution of the clamping action.

[0044] In the embodiments of the present application, please continue to refer to Figure 11 , the air supply device 20 includes a plurality of air supply mechanisms 21, one of which is connected to one of the air nozzles 51. The plurality of air supply mechanisms 21 are respectively provided with independent flow adjusting devices and pressure control devices to realize accurate control of the air supply parameters of each air nozzle 51. Alternatively, the picking mechanical arm 100 further includes a plurality of air valves (not shown in the figure), which are all connected to the air supply device 20. One of the air valves is connected to one of the air nozzles 51, and the plurality of air valves are all connected to the main air source pipeline of the air supply device 20. Each air valve controls the ventilation state of one air nozzle 51. Preferably, the air valve adopts an electromagnetic driving mode, which has fast response speed and can realize millisecond-level opening and closing action. The configuration mode of one air valve connected to one air nozzle 51 ensures the independence and accuracy of control. When the fruits only have blade shielding in a certain direction, the air nozzle assembly 50 can selectively activate the air nozzle 51 in the corresponding direction to avoid unnecessary air flow consumption. When the degree of blade shielding is different, the air nozzle assembly 50 can adjust the air pressure intensity of different air nozzles 51 to realize accurate local air flow control. When the clamping mechanism 30 approaches the fruits, the air nozzle assembly 50 can gradually close unnecessary air nozzles 51 to reduce air flow interference to the fruits and improve the clamping success rate. Through the arrangement of the plurality of air supply mechanisms 21 or the plurality of air valves, accurate control of the air flow and efficient use of energy are realized, which provides a more intelligent and economical solution for complex agricultural picking environment.

[0045] In the embodiments of the present application, please refer to Figure 10The picking mechanical arm 100 further comprises a protection assembly 70 arranged at the end of the mechanical arm body 10, which is used to open or close the plurality of air nozzles 5151, and to protect the plurality of air nozzles 5151 from external impurities when the air nozzle assembly 50 is not working, and to open the air nozzle 51 channel in time when it is needed to be used.

[0046] In particular, please refer to Figure 10 and Figure 11The protection assembly 70 includes a switch driving member 71, a rotating member 72, and a plurality of baffle plates 73. The switch driving member 71 is arranged at the end position of the mechanical arm body 10 and uses a micro motor as a power source. The micro motor has the characteristics of small size, low power consumption, and fast response, and can provide sufficient driving torque in a limited installation space. The rotating member 72 is connected with the switch driving member 71. The rotating member 72 is in a disc-shaped structure and is supported by a bearing at the center of the disc to ensure the stability and accuracy of the rotating movement. The plurality of baffle plates 73 are distributed along the circumference of the rotating member 72. The number of the baffle plates 73 corresponds to the number of the air nozzles 51. The baffle plates 73 are designed in a fan-shaped structure, which can completely cover the openings of the corresponding air nozzles 51 and will not interfere with adjacent baffle plates 73. Preferably, the baffle plates 73 are made of lightweight high-strength engineering plastic, and the surface is treated to improve wear resistance and sealing performance. After the switch driving member 71 receives the control signal, the rotating member 72 is driven to rotate in a preset direction. The rotating member 72 drives the plurality of baffle plates 73 to rotate synchronously. The relative positions between the baffle plates 73 and the air nozzles 51 change. When the baffle plates 73 are staggered with the air nozzles 51, the air outlet channels of the air nozzles 51 are completely exposed, and the compressed air can be smoothly sprayed out. When the baffle plates 73 overlap the air nozzles 51, the baffle plates 73 completely block the openings of the air nozzles 51, preventing airflow from passing through and preventing foreign matter from entering. One baffle plate 73 controls the physical shielding of one air nozzle 51. All the baffle plates 73 realize unified synchronous movement through the rotating member 72. When the rotating member 72 rotates to the position where all the baffle plates 73 are staggered, all the air nozzles 51 are opened at the same time. When the rotating member 72 rotates to the position where all the baffle plates 73 are overlapped, all the air nozzles 51 are closed at the same time. When the air nozzle assembly 50 is not working, it prevents foreign matter from entering all the air nozzles 51. When the system needs to work, it opens all the air nozzle channels at the same time. It should be noted that a plurality of independent air valves are respectively arranged on the air path branches from the main air supply device 20 to the air nozzles 51. Each air valve controls the on-off state of one air path. The air valve adopts an electromagnetic driving mode. The mechanical arm can selectively open or close the air nozzles 51 in a specific direction according to the specific position and degree of the blade shielding. When there is blade shielding on the left side of the fruit, the air supply device 20 only opens the left air nozzles 51 for local treatment. When there is shielding in multiple directions at the same time, the air supply device 20 can simultaneously activate multiple air nozzles 51 and adjust the working intensity respectively. When the clamping mechanism 30 approaches the fruit to perform the clamping action, the air supply device 20 can gradually reduce or close the airflow output to avoid interference of the airflow with the clamping accuracy.

[0047] In the embodiments of the present application, the mechanical shielding mode of the protection assembly 70 provides reliable sealing effect, effectively preventing dust, moisture and plant debris from entering the air nozzles 51. Secondly, the rotating switch mechanism realizes the simultaneous control of the plurality of air nozzles 51, simplifying the complexity of the control system. Finally, the gap between the baffle plates 73 and the air nozzles 51 is accurately controlled, which not only ensures effective shielding, but also avoids excessive friction caused by wear.

[0048] Please read again Figure 10 , the air nozzle assembly 50 is located in the middle of the clamping mechanism 30, the clamping assembly includes a base 31, a plurality of fingers 32 and a clamping drive 33, the base is provided at the end of the mechanical arm body 10, the base 31 adopts a circular or polygonal cross-section design to provide sufficient installation space for the installation and driving mechanism of the fingers 32. The inside of the base 31 is provided with a complete transmission system and control circuit to realize power transmission and signal communication of the mechanical arm body 10. A plurality of fingers 32 are provided on the surface of the base 31 away from the end of the mechanical arm body 10, the number of fingers 32 is usually three or four, and they are distributed on the circumference of the base 31 at equal angles. The fingers 32 adopt an arc or linear design, preferably, the surface of the fingers 32 is covered with soft material with high friction coefficient to ensure stable gripping of the fruit while avoiding surface damage. The fingers 32 are connected to the base 31 through a hinged mechanism and can open and close in the radial plane. The clamping drive 33 is provided inside the base 31 to provide opening and closing power for the plurality of fingers 32. Preferably, the clamping drive 33 adopts a pneumatic or electric drive mode, and the power is transmitted to each finger 32 through a connecting rod mechanism or a gear transmission system. The design of the drive system ensures that all fingers 32 can act synchronously and coordinately, forming enough grabbing space in the open state and providing appropriate clamping force in the closed state. The plurality of fingers 32 are distributed radially around the air nozzle assembly 50, and the gap between the fingers 32 provides an unobstructed jet channel for the airflow. The layout of the annular air nozzle 51 of the air nozzle assembly 50 is coordinated with the annular distribution of the fingers 32 to ensure that the airflow can be smoothly sprayed from the gap between the fingers 32 without being blocked by the mechanical structure.

[0049] When the clamping mechanism 30 is in the open state, the fingers 32 expand outward to form an open grabbing space, and the air nozzle assembly 50 is completely exposed in the central area surrounded by the fingers 32. At this time, the plurality of air nozzles 51 can spray airflow to the surrounding without obstacles, forming a complete leaf pushing open coverage. The airflow spreads outward from the center without interfering or affecting the already opened fingers 32; when the leaves are blown open by the airflow and the fruit is completely exposed, the clamping mechanism 30 starts to perform the clamping action, and the plurality of fingers 32 shrink from the open position to the center to gradually surround the target fruit. During the shrinking process of the fingers 32, the working intensity of the air nozzle assembly 50 can be adjusted accordingly to avoid the influence of too strong airflow on the clamping accuracy. When the fingers 32 completely close to clamp the fruit, the air nozzle assembly 50 stops working, and the whole picking action is completed.

[0050] In the embodiment of the application, the fingers 32 are pneumatic fingers 32, and the clamping drive 33 is a gas pump.

[0051] In the embodiment of the application, please continue to read Figure 13The base 31 is provided with a groove 301 on the end surface away from the mechanical arm body 10, the protection assembly 70 is arranged in the groove 301, the cooperation of the protection assembly 70 and the clamping mechanism 30 is realized, the groove 301 provides an accurate positioning reference for the protection assembly 70, the switch driving part 71, the rotating part 72 and the plurality of baffles 73 can stably work in a preset space range, and the motion interference or function abnormality caused by installation error is avoided. Moreover, the side wall of the groove 301 forms a natural protection barrier, which prevents foreign matters from directly contacting the moving parts of the protection assembly 70, and prolongs the service life of the mechanical structure. The rotating part 72 realizes more stable rotary motion under the constraint of the groove 301, when the protection assembly 70 needs to be checked, cleaned or replaced, the technician can directly contact the related parts by disassembling, such as the cover plate of the groove 301, without disassembling the whole mechanical arm, and the design of the groove 301 provides sufficient operation space for the use of maintenance tools.

[0052] In the embodiment of the present application, the picking mechanical arm 100 further comprises a camera (not shown in the figure) and a controller (not shown in the figure), the camera is arranged at the end of the mechanical arm body 10, and the camera is used to collect images at the to-be-clamped object, preferably, the camera adopts a high-resolution digital image sensor, has automatic focusing and exposure adjustment functions, and can obtain clear fruit and leaf images under different illumination conditions. The camera is configured with a wide-angle lens system, and the field of view covers the complete working area of the clamping mechanism 30, so that each link in the picking process can be comprehensively monitored, the controller is connected with the camera, the clamping mechanism 30 and the gas supply device 20 respectively, the controller is used to identify the exposure information of the to-be-clamped object according to the images, and control the clamping mechanism 30 and the gas supply device 20 according to the exposure information, preferably, the controller is built-in with a high-performance image processing chip and an artificial intelligence algorithm module, has real-time image analysis and decision-making capabilities, the controller is connected with the camera through a special data line to receive continuous high-definition image data streams, and at the same time, the controller establishes bidirectional communication with the clamping mechanism 30 and the gas supply device 20 through a control bus.

[0053] The embodiment of the present application provides a picking mechanical arm 100, which comprises a gas supply device 20, a mechanical arm body 10, a clamping mechanism 30, a swing mechanism 40 and a gas nozzle assembly 50, the clamping mechanism 30 is arranged at the tail end of the mechanical arm body 10, the clamping mechanism is used for clamping a to-be-clamped object, the swing mechanism 40 is arranged on the clamping mechanism 30, the gas nozzle assembly 50 is arranged on the swing mechanism 40, the gas supply device 20 is connected with the gas nozzle assembly 50, the gas supply device 20 is used for supplying gas to the gas nozzle assembly 50, the gas nozzle assembly 50 is used for blowing gas to a shielding object shielding the to-be-clamped object to expose the to-be-clamped object, the clamping mechanism 30 is used for clamping the exposed to-be-clamped object, and the swing mechanism 40 drives the gas nozzle assembly 50 to swing, so as to adjust the angle of blowing gas to the shielding object. By arranging the swing mechanism 40 and the gas nozzle assembly 50 on the clamping mechanism 30, the single-arm cooperative working capability of the mechanical arm is realized, the swing mechanism 40 can drive the gas nozzle assembly 50 to swing at multiple angles, the whole mechanical arm body 10 does not need to be moved, and a wide range of fruit searching can be realized, the working efficiency is improved, and the system energy consumption of the picking mechanical arm 100 is reduced. The gas nozzle assembly 50 adopts an active blowing mode to process the shielding object, effectively solves the technical problem that a traditional visual recognition system cannot penetrate the shielding of leaves, the design of the swing mechanism 40 enables the gas nozzle assembly 50 to flexibly adjust the blowing angle and direction, the effective working range of the picking mechanical arm 100 is expanded, the complex moving operation of the whole picking robot 200 is reduced, and a high-efficiency and reliable technical solution is provided for modern precision agriculture.

[0054] The present application also provides a picking robot embodiment, please refer to Figure 20 The picking robot 200 comprises a movable chassis 201, a loading frame 202 and the picking mechanical arm 100. The movable chassis 201 bears the moving carrier function of the whole picking robot 200, the loading frame 202 is arranged on the movable chassis 201 and is used for collecting and storing picked fruits. The mechanical arm body 10 of the picking mechanical arm 100 is arranged on the movable chassis 201, and the gas supply device 20 is arranged on the movable chassis 201. The picking robot 200 first moves to a target picking area and accurately positions, the mechanical arm performs fruit recognition and picking actions, the gas supply device 20 provides stable gas source support for the gas nozzle assembly 50, and the loading frame 202 receives and stores the picked fruits. When the loading frame 202 approaches a full load state, the picking robot 200 automatically returns to a designated unloading point for fruit transfer.

[0055] For the structure and function of the picking mechanical arm, please refer to the above embodiment, which will not be repeated here.

[0056] Further, the picking robot further comprises a controller, the controller is arranged on the movable chassis, and the controller comprises at least one processor and a memory, the at least one processor is in communication connection with the memory, the clamping mechanism, the gas supply device and the camera device respectively, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the fruit picking method.

[0057] The memory is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the fruit picking method in the embodiments of the present application. The processor executes various function applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory, that is, implements the fruit picking method in the method embodiment.

[0058] The memory can include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the fruit picking device and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the fruit picking device in the following method embodiment through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0059] The one or more modules are stored in the memory, and when executed by the one or more processors, execute the fruit picking method in any method embodiment described above, for example, and execute the fruit picking device described above.

[0060] The above product can execute the method provided in the embodiments of the present application, has the function modules and beneficial effects corresponding to the executed method, and the technical details not described in detail in the embodiments can be referred to the method provided in the embodiments of the present application.

[0061] The embodiments of the present application further provide a non-volatile computer readable storage medium, the computer readable storage medium stores computer executable instructions, the computer executable instructions are executed by one or more processors, for example, execute the steps of the fruit picking method described above, and execute the fruit picking method described below.

[0062] The embodiment of the present application also provides a computer program product, comprising a computer program stored on a non-volatile computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the method of picking fruits in any of the method embodiments described below, for example, perform the above method steps, and perform the fruit picking device described above.

[0063] Please refer to Figure 14 The present application also provides a fruit picking method, the method comprising the following steps: Step S101: determining a hidden fruit hidden in the to-be-picked region, and a hidden position of the hidden fruit; When the picking robot completes the picking of all visible fruits in the to-be-picked region, the search and positioning process of the hidden fruit is started. The camera device comprises an image recognition system and an image acquisition system. The camera device first performs a comprehensive scan on the entire to-be-picked region to obtain complete image information in the region. Through an image recognition algorithm, the system analyzes the region covered by dense leaves and identifies the position where the hidden fruit may exist.

[0064] The picking mechanical arm drives the air supply device to start working and provides air flow to the air nozzle 51 assembly. The picking mechanical arm drives the air nozzle 51 assembly to traverse the to-be-picked region according to a preset path under the instruction of the controller. In the traversal process, the air nozzle 51 assembly continuously supplies air to the dense leaf region, and the camera device acquires image data in real time.

[0065] When the air flow acts on the leaves, the originally blocked fruit may be exposed due to the movement of the leaves. The image recognition system continuously monitors the changes in the field of view, and once the fruit features are detected, the spatial coordinate information of the fruit is immediately recorded to determine the specific position of the hidden fruit.

[0066] Step S102: driving the picking mechanical arm to blow air to the hidden position through the air nozzle 51 assembly to expose the hidden position; After the position of the hidden fruit is determined, the picking robot calculates the motion path of the picking mechanical arm, drives the picking mechanical arm to move, and makes the air nozzle 51 assembly accurately align with the identified hidden position. The joints of the picking mechanical arm move in coordination to ensure that the air nozzle 51 assembly can align with the target position at the best angle and distance.

[0067] The air supply device provides stable air flow to the air nozzle 51 assembly according to preset parameters. The air flow is sprayed out of the air nozzle 51 assembly and directly acts on the leaves around the hidden fruit. The continuous air flow action makes the shielding leaves shift outward, gradually exposing the hidden position.

[0068] During the blowing process, the camera continuously monitors the state changes of the target area. The image processing system analyzes the leaf movement and the degree of fruit exposure. When it detects that the hidden fruit is completely exposed and stable in position, it confirms that the exposure operation is successfully completed.

[0069] Step S103: Drive the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position. After the hidden position is successfully exposed, the picking robot re-plans the motion trajectory of the picking mechanical arm. According to the accurate coordinate information of the exposed fruit, the optimal path from the current position to the target position is calculated. The path planning considers the obstacle avoidance requirement to ensure that the picking mechanical arm does not collide with surrounding branches or other obstacles during the movement.

[0070] The joints of the mechanical arm start to coordinate according to the calculated motion instructions. During the movement, the air supply device continues to work, and the air nozzle 51 assembly continuously supplies air to the target area, keeping the leaves in the offset state to prevent the fruit from being re-shielded.

[0071] The camera continuously tracks the position changes of the target fruit during the movement of the picking mechanical arm. If it detects that the fruit position moves, the system adjusts the motion trajectory of the mechanical arm in time to ensure that the clamping mechanism can accurately reach the fruit position.

[0072] When the clamping mechanism reaches the predetermined position, i.e., the distance from the target fruit meets the clamping operation requirements, the picking mechanical arm stops moving to prepare for subsequent clamping operations.

[0073] Step S104: Drive the clamping mechanism to clamp the hidden fruit, and control the air supply device to stop supplying air. After the clamping mechanism reaches the hidden position, it starts to perform the fruit clamping operation. The clamping fingers of the clamping mechanism automatically adjust the opening according to the size and shape of the fruit to ensure that the fruit can be firmly clamped without causing damage to the fruit.

[0074] The clamping fingers of the clamping mechanism slowly close and gradually surround the target fruit. The force sensor monitors the clamping force, and when it reaches the preset clamping force threshold, it confirms that the fruit has been firmly clamped. At this time, the picking robot immediately issues an instruction to control the air supply device to stop supplying air to the air nozzle 51 assembly.

[0075] After stopping the air supply, the surrounding leaves gradually return to their original positions under the action of gravity and elasticity. Since the fruit has been firmly fixed by the clamping mechanism, the recovery of the leaves will not affect the progress of the picking operation.

[0076] Step S105: Drive the clamping mechanism to recover to pick the hidden fruit.

[0077] After the fruit is successfully clamped, the picking robot drives the clamping mechanism to start the recovery action. The joints of the picking robot move in coordination to drive the clamping mechanism along the predetermined trajectory away from the picking area, avoiding unnecessary contact with the surrounding branches and leaves. During the recovery process, the clamping mechanism maintains stable clamping of the fruit. The force feedback system continuously monitors the clamping state to ensure that the fruit does not fall off or be damaged during transportation.

[0078] When the clamping mechanism moves to the predetermined fruit collection position, the picking operation of a single hidden fruit is completed. The fruit can be placed in the picking basket or transported to the designated collection area. The blowing function of the air nozzle 51 assembly successfully solves the problem of hidden fruits that is difficult to handle by traditional picking methods, improving the comprehensiveness and thoroughness of picking.

[0079] Please refer to Figure 15 In some embodiments, the step S101 of determining the hidden fruit in the picking area and the hidden position of the hidden fruit further comprises: Step S111: After all the exposed fruits in the picking area have been picked, control the air supply device to supply air to the air nozzle 51 assembly; After all the exposed fruits in the picking area have been picked, the picking robot sends a command to control the air supply device to start supplying air to the air nozzle 51 assembly. After receiving the start signal, the air pump or compressor inside the air supply device starts to operate, generating a stable air flow. The air flow is transported from the air supply device to the air nozzle 51 assembly installed at the end of the mechanical arm body through the connecting pipeline. After receiving the air flow, the air nozzle 51 assembly directs the air flow out through its outlet. The initial stage of air flow intensity is set to a preset basic wind level, which can effectively push the blade to move and avoid excessive impact on the fruit.

[0080] The working parameters of the air supply device can be adjusted according to actual needs. The pressure sensor built-in the system monitors the pressure change in the air path in real time to ensure the stability of the air flow output. The temperature sensor monitors the working temperature of the air supply device to prevent overheating of the equipment affecting normal work.

[0081] Step S112: After the air supply device supplies air to the air nozzle 51 assembly, drive the picking robot to make the air nozzle 51 assembly traverse the picking area in sequence; After the air supply device supplies air to the air nozzle 51 assembly, the picking robot drives the picking robot to move, making the air nozzle 51 assembly traverse the picking area in sequence. The motion control system of the mechanical arm calculates the motion angle and speed of each joint according to the preset traversal path planning.

[0082] The traversal process adopts a systematic scanning mode. The picking robot first moves the air nozzle 51 assembly to a boundary position of the picking area, and then moves the air nozzle 51 assembly along a predetermined trajectory at a set interval distance and movement speed. The traversal trajectory can be grid-shaped, spiral-shaped, or other path patterns that can ensure full coverage of the area.

[0083] During the traversal process, the air nozzle 51 assembly continuously sprays air flow. The air flow acts on the leaf-dense area, causing the originally stationary leaves to move. Some fruits that are hidden by the leaves may be exposed due to the movement of the leaves. The movement speed of the picking robot is designed to ensure that there is enough time for the air flow to take effect and maintain reasonable work efficiency.

[0084] Step S113: During the traversal process of the air nozzle 51 assembly in the picking area, according to the image picked up by the camera, it is identified whether there is hidden fruit; The camera works continuously throughout the traversal process to collect image information of the picking area at a preset frame rate. The image processing system analyzes the collected images in real time. The image processing system first pre-processes the images, including noise reduction, contrast enhancement, color correction, etc., to improve image quality. Then use the fruit recognition algorithm to search for the area in the image that meets the characteristics of the target fruit.

[0085] The fruit recognition algorithm is based on color, shape, texture, and other characteristic parameters. The fruit recognition algorithm can distinguish between mature and immature fruits, fruits and leaves, fruits and branches, and other different objects. When the air flow causes the leaves to move, the originally hidden fruit area may change in the image, and the fruit recognition algorithm can capture the new fruit features in the image.

[0086] To improve the accuracy of recognition, the image recognition system uses a continuous frame image comparison analysis method. By comparing the images at adjacent time points, changes in the area are detected. When there is a significant change in fruit features before and after the movement of the leaves at a certain position, the image recognition system determines that there may be hidden fruit at that position.

[0087] Step S114: If there is, locate the hidden position of the hidden fruit.

[0088] When the image recognition system confirms that there is hidden fruit in the picking area, the image recognition system starts to locate the hidden position of the hidden fruit. The position locating process combines image information and the spatial coordinate system of the picking robot.

[0089] The image coordinates acquired by the camera need to be converted into three-dimensional coordinates in the working space of the picking robot. The camera establishes a conversion relationship between the image coordinate system and the picking robot coordinate system using the camera internal parameters and external parameters calibrated in advance. The position of the hidden fruit in the working space of the picking robot is determined through coordinate transformation calculation.

[0090] The image recognition system uses a multi-view image fusion method to improve positioning accuracy. Images of the target area are acquired at different angles, and the spatial coordinates of the fruit are calculated through a three-dimensional reconstruction algorithm. The distance information provided by the depth sensor further improves the accuracy of position measurement. The image recognition system stores the determined position information of the hidden fruit in the memory, providing target coordinates for subsequent picking operations. The position information includes three-dimensional coordinates, orientation angle, size estimation, and other parameters, providing necessary data support for the planning of the clamping operation of the clamping mechanism.

[0091] In the embodiments of the present application, when the hidden position is fully exposed, the system performs the subsequent steps of step S101. The picking robot moves towards the hidden position until the clamping mechanism reaches the hidden position. During the movement of the picking robot, the air supply device continues to work, maintaining the deflection state of the leaves to prevent the fruit from being re-occluded. After the clamping mechanism reaches the target position, it starts to perform the clamping action. The clamping fingers of the clamping mechanism automatically adjust the opening according to the size of the fruit, and then slowly close to clamp the fruit. The force sensor monitors the clamping force to ensure that the fruit is firmly clamped without causing damage to the fruit. When the clamping mechanism successfully clamps the hidden fruit, the control system of the picking robot immediately controls the air supply device to stop supplying air. After stopping air supply, the surrounding leaves gradually return to the natural state under the action of gravity and elasticity. Finally, the picking robot drives the clamping mechanism to recover, completing the picking of the hidden fruit. The clamping mechanism moves along the predetermined trajectory to the collection position with the picked fruit, and places the fruit into the picking basket or designated collection area, completing the entire picking cycle.

[0092] Please refer to Figure 16 In some embodiments, the step S102 of driving the picking robot to blow air at the hidden position with the air nozzle 51 assembly to expose the hidden position includes further comprising: Step S211: driving the picking robot to make the air nozzle 51 assembly face the hidden position; When the picking robot determines the position coordinates of the hidden fruit, it sends a movement command to drive the picking arm to move towards the target position. The multiple joints of the picking arm move in coordination according to the calculated movement trajectory, enabling the air nozzle 51 assembly to accurately reach the predetermined position. During the movement of the picking arm, the position feedback sensor monitors the angle changes of each joint in real time, ensuring the accuracy of the movement trajectory. When the air nozzle 51 assembly approaches the hidden position, the picking robot switches to a precise positioning mode, reducing the movement speed and improving the position control accuracy. Finally, the picking arm drives the air nozzle 51 assembly to reach the optimal blowing position opposite the hidden position. The position control system confirms that the air outlet direction of the air nozzle 51 assembly forms an appropriate angle and distance with the leaf area around the hidden fruit, creating optimal conditions for subsequent blowing operations.

[0093] Step S212: Control the air supply device to supply air to the air nozzle 51 assembly at a preset wind level; After the picking arm control system completes the positioning of the air nozzle 51 assembly, it immediately controls the air supply device to supply air to the air nozzle 51 assembly at a preset wind level. The preset wind level is pre-set according to factors such as fruit tree variety, leaf density, seasonal environment, etc., and usually selects moderate intensity as the initial wind force.

[0094] After receiving the wind level instruction, the air supply device adjusts the internal air flow control valve and compressor speed to produce air flow output of corresponding intensity. The air flow is transmitted to the air nozzle 51 assembly through the connecting pipeline and is directed to spray from the air nozzle 51 outlet.

[0095] It should be noted that the selection of the initial wind level is of great significance. Too small wind force may not be able to effectively push the leaves to expose the fruit, and too large wind force may cause the fruit to swing or fall off. The preset wind level serves as an empirical reference value, providing a reasonable starting point for subsequent dynamic adjustment.

[0096] Step S213: Identify whether the hidden fruit is exposed after a preset duration; The air nozzle 51 assembly starts to blow air to the hidden position at a preset wind level. The timer built into the picking arm starts to record the blowing duration, and when the preset duration is reached, the exposure detection program is automatically started. The setting of the preset duration needs to consider the time characteristics of the leaf response to air flow, ensuring enough time for the air flow to take effect.

[0097] During the whole blowing process, the camera continuously captures image information of the target area. The image processing system analyzes the captured images in real time, focusing on monitoring changes in the hidden position area. When the leaves move under the action of the airflow, the originally hidden fruit area may gradually be exposed. The exposure state detection algorithm identifies the newly appearing fruit features in the target area by comparing the differences between the images before and after blowing. The algorithm uses shape recognition, color analysis, edge detection, and other technical means to improve the accuracy and reliability of fruit exposure state judgment.

[0098] Step S214: If the hidden fruit is not exposed, determine whether the air supply device has reached the maximum wind level; When the preset time period ends, the image recognition system immediately performs identification and judgment of the exposure state of the hidden fruit. If the image analysis result shows that the hidden fruit has not been fully exposed, the image recognition system enters the wind level adjustment program.

[0099] First, the control system of the picking robot arm checks whether the wind level of the current air supply device has reached the preset maximum wind level. The air supply device has multiple adjustable wind levels, each corresponding to a different airflow intensity output. The maximum wind level represents the maximum airflow intensity that the air supply device can provide within the safe working range.

[0100] Step S215: If the air supply device has not reached the maximum wind level, increase the wind level of the air supply device by one level, and return to the step of identifying whether the hidden fruit is exposed after a preset time period, until the exposure of the hidden fruit is identified; If the current wind level has not reached the maximum value, the control system of the picking robot arm performs the wind level increase operation. The controller of the picking robot arm sends a wind adjustment instruction to the air supply device, increasing the wind level by one level from the current level. After responding to the instruction, the air supply device increases the airflow output intensity, and the air nozzle 51 assembly sprays stronger airflow onto the target area.

[0101] After the wind level is increased, the control system of the picking robot arm restarts the timing and continues the blowing operation at the new wind level for a preset time period. After the time period ends, the exposure state detection of the hidden fruit is performed again. If the fruit is still not exposed and the wind level has not reached the maximum value, the system will again perform the wind level increase, forming a cyclic adjustment process.

[0102] Step S216: If the hidden fruit is exposed, fix the current wind level of the air supply device.

[0103] When the system detects that the hidden fruit is successfully exposed, the wind level fixing operation is immediately performed. The current wind level of the air supply device is fixed, and further wind adjustment is stopped to ensure the stable maintenance of the exposure state.

[0104] In the embodiment of the present application, the step S217 of sending a picking failure alarm information to an external device is further included, wherein the picking failure alarm information contains the hidden position of the hidden fruit.

[0105] When the wind level reaches the preset maximum wind level and the fruit is still not exposed, the picking robot control system activates the picking failure alarm mechanism. The triggering of the alarm mechanism indicates that the current degree of hiding the fruit exceeds the range that can be effectively handled by the airflow blowing.

[0106] The picking robot control system generates picking failure alarm information, and the information content contains the specific hidden position of the hidden fruit that leads to picking failure. The position information is expressed in three-dimensional coordinates of the robot working coordinate system, ensuring that the maintenance personnel or operators can accurately locate the problem area.

[0107] The alarm information is sent to the external device through the communication interface. The external device can be a remote monitoring system, a mobile terminal device or a human-computer interaction interface. The timely delivery of the alarm information enables relevant personnel to understand the difficulties encountered in the picking operation and take appropriate manual intervention measures.

[0108] In the embodiment of the present application, the step S218 of controlling the air supply device to supply air to the air nozzle 51 assembly at a preset wind level for a preset time length, and then controlling the air supply device to gradually reduce the wind level until the minimum wind level that can expose the hidden fruit is found, is further included.

[0109] After confirming that the fruit is exposed, the picking robot can also perform a wind optimization program. The picking mechanical arm controls the air supply device to gradually reduce the wind level to find the minimum wind level that can maintain the exposure of the hidden fruit. The goal of wind optimization is to reduce the airflow intensity as much as possible under the premise of ensuring sufficient exposure of the fruit, and to reduce unnecessary impact on the fruit. The wind reduction process adopts a step-by-step decreasing manner. The control system of the picking mechanical arm reduces the current wind level by one level, and then monitors whether the exposure state of the fruit can still be maintained. If the fruit remains exposed, the control system of the picking mechanical arm continues to reduce the wind level; if it is detected that the fruit begins to be re-shielded, the control system of the picking mechanical arm immediately restores the wind level to the previous level, and determines that this level is the minimum effective wind level.

[0110] In some embodiments, after the air nozzle 51 assembly is successfully exposed from the hidden position, the control system of the picking robot starts to execute the motion control program of the picking manipulator towards the hidden position. Before officially starting the motion, the picking robot first confirms the working status of each component. The air supply device continues to supply air to the air nozzle 51 assembly at a certain wind level, maintaining the deflection state of the blade and the exposure condition of the fruit. The camera adjusts to the optimal shooting angle and parameter setting, ensuring that the movement state of the fruit in the target area can be clearly captured. The image processing system switches to a high-precision dynamic detection mode, improving the sensitivity and recognition accuracy of small motion changes. The joint drivers of the picking manipulator receive motion instructions and start to move in coordination according to the pre-calculated trajectory path. The motion speed is set at a moderate level to ensure picking efficiency and provide sufficient reaction time for dynamic monitoring. Please refer to Figure 17 , the step S103 of driving the picking manipulator to move towards the hidden position until the clamping mechanism reaches the hidden position, further comprising: Step S131: During the movement of the clamping mechanism towards the hidden position, identify whether the hidden fruit swings according to the image captured by the camera; The swing recognition algorithm is based on image sequence analysis technology, which judges the swing state by tracking the position change of the fruit in consecutive frames. The algorithm first establishes a feature model of the fruit, including color distribution, shape contour, texture features, and other recognition parameters. Motion detection uses a combination of optical flow and background difference techniques. The optical flow method calculates the motion vector of the pixel points in the image, which can accurately describe the motion direction and speed of the fruit. The background difference method highlights the outline and range of the moving target by comparing with the static background.

[0111] When the detection algorithm identifies that the fruit has a position offset, the image recognition system further analyzes the motion characteristics. By calculating the periodicity and amplitude characteristics of the position change, it distinguishes between swing motion and other types of displacement. Swing motion usually exhibits reciprocating motion around the equilibrium position, with obvious periodic characteristics.

[0112] During the movement of the clamping mechanism to the hidden position, it is identified whether the hidden fruit swings according to the image collected by the camera. The camera continuously collects a sequence of images of the target region in high frame rate mode, ensuring that any movement of the fruit can be captured. The image collection process uses multiple exposure modes to adapt to different lighting conditions. The automatic focusing system continuously tracks the fruit position to ensure that the image always maintains clarity. The image stabilization algorithm compensates for the vibration caused by the mechanical arm movement to improve the consistency of image quality. The data processing of continuous image monitoring uses a real-time stream processing architecture. Each frame of image is pre-processed and feature extracted quickly, focusing on analyzing the pixel changes in the fruit area. The moving target in the image is detected by the frame difference algorithm to identify the change trajectory of the fruit position.

[0113] Step S132: If it is identified that the hidden fruit swings, the maximum swing amplitude of the hidden fruit is identified in real time; Once it is confirmed that the hidden fruit swings, the image recognition system immediately starts the real-time swing amplitude identification program. The calculation of the swing amplitude is based on the position trajectory data of the fruit center point in the image coordinate system. The image recognition system continuously tracks the coordinate changes of the fruit center point and records the maximum offset distance during the swing process. The measurement of the swing amplitude uses a three-dimensional space calculation method. Through the calibration parameters of the camera, the pixel coordinates on the image plane are converted into the actual distance in the mechanical arm workspace. The acquisition of depth information combines binocular vision technology and structured light ranging method to ensure the accuracy of distance measurement. The determination of the maximum swing amplitude needs to consider the complete period of the swing motion. The system continuously monitors multiple swing periods and records the maximum distance of the fruit from the equilibrium position in each period. Through statistical analysis method, the representative maximum swing amplitude value is determined.

[0114] Step S133: Determine whether the maximum swing amplitude is greater than the maximum opening of the clamping mechanism; After the swing amplitude measurement is completed, the image recognition system compares and analyzes the measurement results with the maximum opening of the clamping mechanism. The maximum opening of the clamping mechanism refers to the maximum distance between the two clamping fingers of the clamping mechanism in the fully open state, representing the maximum target size that the clamping mechanism can handle.

[0115] The comparison and evaluation process considers the dynamic characteristics of the swing motion. The fruit not only has a position offset during the swing, but also may have a change in posture. The system analyzes the spatial distribution characteristics of the fruit swing trajectory and calculates the effective area range that the clamping mechanism needs to cover. The setting of safety margin ensures the reliability of the clamping operation. Even if the swing amplitude is slightly smaller than the maximum opening of the clamping mechanism, the system will reserve a proper safety gap to avoid clamping failure due to the uncertainty of the swing motion.

[0116] Step S134: When the maximum swing amplitude is greater than the maximum opening of the clamping mechanism, an external device is sent a picking failure warning message; When the maximum swing amplitude is greater than the maximum opening of the clamping mechanism, the image recognition system determines that reliable clamping operation cannot be completed under the current conditions, and the generation of the warning message includes multiple key data items. First, the accurate position coordinates of the hidden fruit causing the warning are recorded, using the standard format of the mechanical arm working coordinate system. Second, the comparison result of the detected maximum swing amplitude value and the maximum opening of the clamping mechanism is recorded.

[0117] It should be noted that the picking failure warning message also includes an image of the location of the hidden fruit. The image is collected at the moment when the fruit swing amplitude is the largest, ensuring that the motion state of the fruit can be clearly displayed. The hidden position is clearly marked in the image, and a high-contrast marker symbol is used to highlight the target area.

[0118] Step S135: When the maximum swing amplitude is less than the maximum opening of the clamping mechanism, continue to drive the clamping mechanism until the clamping mechanism reaches the hidden position.

[0119] When the maximum swing amplitude is less than the maximum opening of the clamping mechanism, the image recognition system confirms that the current swing state will not affect the successful execution of the clamping operation. The control system of the picking robot continues to drive the clamping mechanism to move towards the hidden position until the clamping mechanism reaches the predetermined clamping position.

[0120] During the movement process, the swing monitoring program remains active. The image recognition system continuously monitors the swing state changes of the fruit to prevent sudden increases in swing amplitude caused by external factors. If the swing amplitude exceeds the safety threshold during the movement process, the system can stop the movement in time and re-evaluate the operation conditions.

[0121] In the embodiments of the present application, based on the feedback information of the swing monitoring, the picking robot can dynamically adjust the operation parameters to improve the picking effect. When a slight swing is detected, the picking robot can appropriately adjust the airflow direction and intensity to reduce the disturbance effect on the fruit. The movement speed of the picking robot can also be adaptively adjusted according to the swing situation. When the fruit swings violently, reducing the movement speed can reduce the additional vibration caused by the movement of the mechanical arm. When the fruit remains relatively stable, appropriately increasing the movement speed can improve the picking efficiency.

[0122] In some embodiments, referring to Figure 18 Before step S106: driving the picking robot to move towards the hidden position until the clamping mechanism reaches the hidden position, the method further comprises: Step S106: When the air nozzle 51 assembly is directly opposite the hidden position and blows air to expose the hidden position, whether the hidden fruit swings is identified according to the image captured by the camera; When the air nozzle 51 assembly starts to blow air directly opposite the hidden position to expose the hidden position, the camera starts to monitor the swing state synchronously, and the camera starts to continuously capture images at the beginning of the air flow. The capture program is set to high-precision mode to ensure that the small position changes of the fruit under the influence of the air flow can be accurately captured. The image capture frequency is optimized according to the expected swing frequency range, which meets the time resolution requirements of motion detection and ensures the real-time processing capability of the data processing system.

[0123] Step S107: If it is identified that the hidden fruit swings, the maximum swing amplitude of the hidden fruit is identified in real time; When the system detects that the position of the fruit continuously changes, the swing feature analysis algorithm starts to work. The algorithm establishes the trajectory data of the position change with time by calculating the position sequence of the center point of the fruit in the continuous image frames.

[0124] Step S108: Whether the swing amplitude of the hidden fruit is greater than the maximum opening of the clamping mechanism is judged, and if so, the picking failure alarm information is sent to the external device; When the judgment result shows that the swing amplitude of the hidden fruit is greater than the maximum opening of the clamping mechanism, the picking robot determines that the clamping operation cannot be successfully executed under the current condition, and the picking mechanical arm immediately activates the picking failure alarm mechanism and sends the picking failure alarm information to the external device.

[0125] The construction of the alarm information contains a plurality of key data items, including the detected maximum swing amplitude value, swing frequency, duration and other parameters, and the maximum opening specification data of the clamping mechanism is also provided, so that the receiver can understand the judgment basis.

[0126] The picking mechanical arm records the accurate position of the hidden fruit causing the alarm in the mechanical arm working coordinate system, including three-dimensional coordinates and attitude angles. The position accuracy meets the needs of subsequent manual intervention or device parameter adjustment.

[0127] The alarm information is sent to the external device through the pre-configured communication interface. The communication protocol supports multiple transmission methods, including Ethernet connection, wireless network, serial interface, etc. The information format adopts a standardized data structure to ensure that different types of receiving devices can correctly parse the alarm content.

[0128] Preferably, the sender waits for the confirmation reply of the receiver after completing the information transmission, verifies whether the information is successfully received and processed. If no confirmation signal is received within the preset time, the system automatically starts the retransmission mechanism to ensure the reliable delivery of the alarm information.

[0129] Step S109: If the swing amplitude of the hidden fruit is less than the maximum opening of the clamping mechanism, proceed to the step of driving the picking mechanical arm to move towards the hidden position.

[0130] When the comparison result shows that the swing amplitude of the hidden fruit is less than the maximum opening of the clamping mechanism, the picking mechanical arm confirms that the current swing state will not hinder the successful execution of the clamping operation. After the judgment, the control program proceeds to the next execution step, that is, driving the picking mechanical arm to move towards the hidden position.

[0131] In the embodiments of the present application, through the complete swing detection and evaluation method described above, the picking robot accurately judges the feasibility of the operation before the picking mechanical arm moves and executes, effectively avoids unnecessary operation attempts and potential failures, and significantly improves the overall efficiency and success rate of the picking operation.

[0132] The present application provides a fruit picking method applied to a picking robot, the picking robot comprising a movable chassis and a picking mechanical arm, the picking mechanical arm comprising a mechanical arm body, an air nozzle 51 assembly, a gas supply device, a camera device and a clamping mechanism, the mechanical arm body and the gas supply device being arranged on the movable chassis, the clamping mechanism and the air nozzle 51 assembly being arranged at the end of the mechanical arm body, the gas supply device being connected with the air nozzle 51 assembly, the camera device being arranged on the picking mechanical arm, the camera device being used to take images of the region to be picked, characterized in that the method comprises: determining a hidden fruit hidden in the region to be picked, and a hidden position of the hidden fruit, driving the picking mechanical arm to make the air nozzle 51 assembly blow air directly at the hidden position to expose the hidden position, driving the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position, driving the clamping mechanism to clamp the hidden fruit, and controlling the gas supply device to stop supplying air, and driving the clamping mechanism to recover to pick the hidden fruit. The design of a single mechanical arm cooperating with the air nozzle 51 assembly in the embodiments of the present application effectively solves the technical problems of high cost and complex control of traditional double-arm picking robots. Compared with the traditional scheme which needs to cooperatively control two mechanical arms, only one mechanical arm cooperating with the air nozzle 51 assembly is needed in the present application to realize the search and picking of hidden fruits. By driving the air nozzle 51 assembly to search the region to be picked by the mechanical arm, the fruits hidden by leaves can be accurately found and positioned, the adaptability of the mechanical arm picking to complex environments is improved, the comprehensiveness and thoroughness of the picking operation are ensured, and the problem of missing hidden fruits is effectively avoided.

[0133] The present application also provides an embodiment of a fruit picking device 80 applied to the above-mentioned picking robot. Please refer to Figure 19The fruit picking device 80 comprises a determining module 81, a first driving module 82, a second driving module 83, a third driving module 84 and a recycling module 85.

[0134] The determining module 81 is configured to determine a hidden fruit in the picking area and a hidden position of the hidden fruit. The first driving module 82 is configured to drive the picking mechanical arm to make the air nozzle 51 component blow air directly at the hidden position to expose the hidden position. The second driving module 83 is configured to drive the picking mechanical arm to move towards the hidden position until the clamping mechanism reaches the hidden position. The third driving module 84 is configured to drive the clamping mechanism to clamp the hidden fruit and control the air supply device to stop supplying air. The recycling module 85 is configured to drive the clamping mechanism to recycle to pick the hidden fruit.

[0135] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, which is made by using the content of the specification and drawings of the present application, is also included in the patent protection scope of the present application.

Claims

1. A fruit harvesting method applied to a harvesting robot, the harvesting robot comprising a movable chassis and a harvesting robotic arm, the harvesting robotic arm comprising a robotic arm body, an air nozzle assembly, an air supply device, a camera device, and a gripping mechanism, the robotic arm body and the air supply device being disposed on the movable chassis, the gripping mechanism and the air nozzle assembly being disposed at the end of the robotic arm body, the air supply device being connected to the air nozzle assembly, the camera device being disposed on the harvesting robotic arm, the camera device being used to capture images of the area to be harvested, characterized in that... The method includes: Identify the hidden fruits within the area to be harvested, and the location where the hidden fruits are hidden; Drive the harvesting robotic arm so that the air nozzle assembly blows air directly at the hidden location to expose the hidden location; Drive the harvesting robotic arm toward the hidden position until the gripping mechanism reaches the hidden position; Drive the clamping mechanism to clamp the hidden fruit, and control the air supply device to stop supplying air; The clamping mechanism is driven to retract, so as to harvest the hidden fruit.

2. The fruit harvesting method according to claim 1, characterized in that, The step of driving the harvesting robotic arm to blow air into the hidden location so as to expose the hidden location includes: Drive the harvesting robotic arm so that the air nozzle assembly is facing the hidden position; The air supply device is controlled to supply air to the nozzle assembly at a preset wind speed level; After a preset time period, it continuously identifies whether hidden fruits are exposed; If the hidden fruit is not exposed, then identify whether the air supply device has reached the maximum wind speed. If the air supply device does not reach the maximum wind force level, the wind force level of the air supply device will be increased by one level, and the process will return to the step of identifying whether the hidden fruit is exposed after a preset duration, until the hidden fruit is identified as exposed. If the hidden fruit is exposed, the current wind speed of the air supply device is fixed.

3. The fruit harvesting method according to claim 2, characterized in that, The method further includes: If the wind force reaches the preset maximum wind force and the fruit is still not exposed, a harvesting failure alarm message is sent to the external device. The harvesting failure alarm message includes the hidden location of the hidden fruit.

4. The fruit harvesting method according to claim 2, characterized in that, The method includes: After the air supply device supplies air to the nozzle assembly at a preset wind speed level for a preset duration, and the hidden fruit is detected to be exposed, the air supply device is controlled to gradually reduce the wind speed until the minimum wind speed level that can expose the hidden fruit is found.

5. The fruit harvesting method according to claim 1, characterized in that, The method further includes driving the harvesting robotic arm toward the hidden position until the gripping mechanism reaches the hidden position: When the air nozzle assembly blows air into the hidden location to expose it, the camera device identifies whether the hidden fruit is swaying based on the image captured by the camera device. If the hidden fruit swings, then determine the amplitude of the swing of the hidden fruit; Determine whether the swing amplitude of the hidden fruit is greater than the maximum opening of the clamping mechanism; if so, send a harvesting failure alarm message to the external device. If the swing amplitude of the hidden fruit is less than the maximum opening of the clamping mechanism, then proceed to the step of driving the picking robotic arm to move toward the hidden position.

6. The fruit harvesting method according to claim 1, characterized in that, The step of driving the harvesting robotic arm to move toward the hidden position until the gripping mechanism reaches the hidden position further includes: During the movement of the clamping mechanism toward the hidden position, the camera device identifies whether the hidden fruit is swaying based on the image captured by the camera device. If the hidden fruit is detected to be swaying, the maximum swaying amplitude of the hidden fruit is identified in real time. Determine whether the maximum swing amplitude is greater than the maximum opening of the clamping mechanism; When the maximum swing amplitude is greater than the maximum opening of the clamping mechanism, a harvesting failure alarm message is sent to the external device; When the maximum swing amplitude is less than the maximum opening of the clamping mechanism, the clamping mechanism continues to be driven until the clamping mechanism reaches the hidden position.

7. The fruit harvesting method according to claim 3, 5, or 6, characterized in that, The harvesting failure alarm information also includes an image of the location of the hidden fruit, with the hidden location marked in the image.

8. The fruit harvesting method according to any one of claims 1-6, characterized in that, The steps of determining the hidden fruit hidden within the area to be harvested, and the location of the hidden fruit, further include: After all the fruits exposed in the harvesting area have been harvested, the air supply device is controlled to supply air to the air nozzle assembly. After the air supply device supplies air to the air nozzle assembly, it drives the harvesting robotic arm so that the air nozzle assembly sequentially traverses the harvesting area. As the air nozzle assembly sequentially traverses the harvesting area, it identifies whether there are hidden fruits based on the images captured by the camera device. If it exists, then locate the hidden location of the hidden fruit.

9. A fruit-harvesting device applied to a harvesting robot, the harvesting robot comprising a movable chassis, a harvesting robotic arm, an air nozzle assembly, an air supply device, a camera device, and a gripping mechanism, wherein the harvesting robotic arm and the air supply device are both disposed on the movable chassis, the gripping mechanism and the air nozzle assembly are both disposed at the end of the harvesting robotic arm, the air supply device is connected to the air nozzle assembly, and the camera device is disposed on the harvesting robotic arm, the camera device being used to capture images of the area to be harvested, characterized in that... The fruit harvesting device includes: The determining module is used to determine hidden fruits hidden in the area to be picked, and the hiding location of the hidden fruits; The first drive module is used to drive the harvesting robotic arm so that the air nozzle assembly blows air towards the hidden position to expose the hidden position; The second drive module is used to drive the picking robotic arm to move toward the hidden position until the gripping mechanism reaches the hidden position; The third drive module is used to drive the clamping mechanism to clamp the hidden fruit and control the air supply device to stop supplying air. A recovery module is used to drive the clamping mechanism to recover and harvest the hidden fruit.

10. A harvesting robot, characterized in that, The harvesting robot includes: Movable chassis; A harvesting robotic arm includes a robotic arm body, a gripping mechanism, an air nozzle assembly, and an air supply device. The robotic arm body and the air supply device are both mounted on a movable chassis. The air nozzle assembly and the gripping mechanism are both located at the end of the robotic arm body. The air supply device is connected to the air nozzle assembly and is used to supply air to the air nozzle assembly. A camera device is installed on the harvesting robotic arm. A controller, mounted on a movable chassis, includes at least one processor and a memory, wherein the at least one processor is communicatively connected to the memory, a clamping mechanism, an air supply device, and a camera device. The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 7.

Images