Fruit picking auxiliary positioning visual sound-light alarm method and system

Abstract

The invention relates to the field of fruit picking, and discloses a fruit picking auxiliary positioning visual sound-light alarm method and system.The method comprises the steps that picking equipment is controlled to drive a tail end executing mechanism to ascend to the needed position, and a radar ranging alarm device, a visual image device and an angle collecting device on the tail end executing mechanism are started; according to the alignment mark line provided by the visual image device, the picking device is controlled to drive the end execution mechanism to align to the fruit tree trunk; real-time angle information is collected, and posture adjustment is conducted on the tail end executing mechanism according to the real-time angle information; the radar ranging alarm device is used for detecting the distance between the end executing mechanism and the fruit tree trunk in real time, corresponding alarm is triggered and generated according to the distance, and the picking equipment is controlled to drive the end executing mechanism to quickly and accurately move towards the fruit tree trunk. The device has the advantages that an operator can conveniently operate a tail end executing mechanism to achieve precise positioning at high altitude, the picking operation efficiency is improved, and collision damage to trunks is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fruit picking, in particular to a fruit picking auxiliary positioning visual sound and light alarm method and system. BACKGROUND

[0002] After the oil palm fruit grows and matures, the oil palm fruit needs to be picked from the oil palm tree. With the continuous development of fruit automatic picking technology, the original manual picking method cannot meet the modern production requirements, and the picking work is continuously advancing towards automation and intelligence, and automatic picking equipment has also appeared, such as the high-altitude picking equipment disclosed in Chinese Patent Application No. CN202411942008.X.

[0003] Because the mechanized picking of oil palm fruit is generally above 10 meters high, the end effector of the picking equipment is in a state of ten meters or higher, and the variable amplitude telescopic arm is in a 70-degree upward state. In this state, the operator in the cab usually has difficulty understanding the specific posture of the end effector and the actual distance between the end effector and the trunk of the fruit tree, which often leads to collisions with the trunk of the fruit tree during the actual picking process, thereby causing damage to the trunk of the fruit tree. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a fruit picking auxiliary positioning visual sound and light alarm method and system, which can facilitate the operator to operate the end effector to achieve precise positioning in the high altitude, effectively reduce the collision between the end effector and the trunk of the fruit tree during the picking process, and thereby reduce the damage to the trunk of the fruit tree caused by the collision.

[0005] To achieve the above purpose, the present application is realized by the following technical solutions:

[0006] In a first aspect, a fruit picking auxiliary positioning visual sound and light alarm method is provided, which comprises the following steps:

[0007] The variable amplitude telescopic arm of the picking equipment is controlled to drive the end effector to rise to the desired position, and the radar ranging alarm device, visual image device and angle acquisition device on the end effector are started;

[0008] According to the alignment mark line provided by the visual image device, the picking equipment is controlled to drive the end effector to align with the trunk of the fruit tree;

[0009] Real-time angle information is collected, and the end effector is adjusted in posture according to the real-time angle information, so that the end effector can approach the trunk of the fruit tree in a horizontal posture;

[0010] The radar distance alarm device is used to detect the distance between the end effector and the fruit tree trunk in real time, and the corresponding alarm is triggered according to the detected distance between the end effector and the fruit tree trunk, and the picking device drives the end effector to move quickly and accurately towards the fruit tree trunk.

[0011] Further, the corresponding alarm triggered according to the distance between the end effector and the fruit tree trunk is specifically that the alignment mark line is divided into at least three identification areas from one end close to the end effector to one end away from the end effector, and the corresponding radar alarm data is set for each identification area in advance; the identification area where the fruit tree trunk is located is determined according to the detected distance between the end effector and the fruit tree trunk, and the corresponding alarm is triggered according to the radar alarm data corresponding to the identification area.

[0012] Further, the corresponding alarm triggered according to the distance between the end effector and the fruit tree trunk is further specifically that:

[0013] The alignment mark line is divided into a dangerous identification area, a close identification area and a safe identification area from one end close to the end effector to one end away from the end effector, the dangerous identification area is provided with first radar alarm data, the close identification area is provided with second radar alarm data, and the safe identification area is provided with third radar alarm data;

[0014] When the fruit tree trunk is determined to be in the safe identification area according to the detected distance between the end effector and the fruit tree trunk, the corresponding alarm is triggered according to the third radar alarm data, and the picking device drives the end effector to move quickly towards the fruit tree trunk;

[0015] When the fruit tree trunk is determined to be in the close identification area according to the detected distance between the end effector and the fruit tree trunk, the corresponding alarm is triggered according to the second radar alarm data, and the picking device drives the end effector to move towards the fruit tree trunk at a reduced speed;

[0016] When the fruit tree trunk is determined to be in the dangerous identification area according to the detected distance between the end effector and the fruit tree trunk, the corresponding alarm is triggered according to the first radar alarm data, and the picking device stops driving the end effector to move towards the fruit tree trunk.

[0017] Further, the real-time angle information is specifically collected by:

[0018] The real-time angle information is collected by the angle collection device and is transmitted back to the cab of the picking device for display;

[0019] Or the visual image device is used to collect the real-time angle information displayed on the angle collecting device, and the real-time angle information is returned to the cab of the picking equipment for display.

[0020] Further, one side of the end effector is provided with a clamping jaw for clamping the trunk of the fruit tree, a radar ranging alarm device is arranged at the middle position of the top of the end effector, a visual image device is arranged at the middle position of the top of the end effector, the visual image device is located away from the clamping jaw, and the shooting end of the visual image device faces the opening of the clamping jaw, and an angle collecting device is arranged at the top of the end effector.

[0021] In the second aspect, a fruit picking auxiliary positioning visual sound and light alarm system is provided, and the system comprises a starting module, an alignment module, a posture adjusting module and a motion alarm module.

[0022] The starting module is used to control the variable amplitude telescopic arm of the picking equipment to drive the end effector to rise to a required position, and to start the radar ranging alarm device, the visual image device and the angle collecting device on the end effector.

[0023] The alignment module is used to control the picking equipment to drive the end effector to align with the trunk of the fruit tree according to the alignment mark line provided by the visual image device.

[0024] The posture adjusting module is used to collect real-time angle information, and to adjust the posture of the end effector according to the real-time angle information, so that the end effector can approach the trunk of the fruit tree in a horizontal posture.

[0025] The motion alarm module is used to use the radar ranging alarm device to detect the distance between the end effector and the trunk of the fruit tree in real time, and to trigger corresponding alarms according to the detected distance between the end effector and the trunk of the fruit tree, and to control the picking equipment to drive the end effector to move quickly and accurately towards the trunk of the fruit tree.

[0026] Further, the triggering of corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is specifically that the alignment mark line is divided into at least three mark areas from one end close to the end effector to one end away from the end effector, and corresponding radar alarm data is set for each mark area in advance; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area.

[0027] Further, the triggering of corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is further specifically that:

[0028] The alignment marking line is divided into a danger marking area, an approach marking area, and a safety marking area from the end closest to the end of the end actuator to the end furthest from the end actuator. The danger marking area is equipped with a first radar alarm data, the approach marking area is equipped with a second radar alarm data, and the safety marking area is equipped with a third radar alarm data.

[0029] When the distance between the detected end effector and the tree trunk is determined to be within the safety marking area, the corresponding alarm is triggered based on the third radar alarm data, and the harvesting equipment is controlled to drive the end effector to move quickly toward the tree trunk.

[0030] When the distance between the detected end effector and the tree trunk is determined to be close to the marked area, the corresponding alarm is triggered based on the second radar alarm data, and the harvesting equipment is controlled to drive the end effector to decelerate towards the tree trunk.

[0031] When the distance between the detected end effector and the tree trunk is determined to be within a danger zone, a corresponding alarm is triggered based on the first radar alarm data, and the harvesting equipment is controlled to stop moving the end effector toward the tree trunk.

[0032] Furthermore, the acquisition of real-time angle information specifically includes:

[0033] The angle acquisition device collects real-time angle information and transmits it back to the cab of the harvesting equipment for display.

[0034] Alternatively, a visual imaging device can be used to collect real-time angle information displayed on the angle acquisition device, and the real-time angle information can be transmitted back to the cab of the harvesting equipment for display.

[0035] Furthermore, one side of the end effector is provided with a gripper for holding the trunk of the fruit tree, and the radar ranging alarm device is located at the middle position of the top of the end effector; the visual imaging device is located at the middle position of the top of the end effector, and the visual imaging device is located on the side away from the gripper, with the shooting end of the visual imaging device facing the opening of the gripper; the angle acquisition device is located at the top of the end effector.

[0036] By adopting the technical scheme of the present application, at least the following beneficial effects are achieved: by arranging the radar ranging alarm device, the visual image device and the angle acquisition device on the end executing mechanism, and in the specific picking process, first, the end executing mechanism is aligned by using the alignment mark line provided by the visual image device, that is, the fruit tree trunk is located between the alignment mark lines on both sides of the picture; then, the real-time angle information is collected, and the posture of the end executing mechanism is adjusted according to the real-time angle information, so that the end executing mechanism can approach the fruit tree trunk in a horizontal posture; finally, the distance between the end executing mechanism and the fruit tree trunk is detected in real time by using the radar ranging alarm device, and the corresponding alarm is triggered according to the distance; therefore, by adopting the technical scheme of the present application, the operator can very conveniently operate the end executing mechanism to realize accurate positioning in the air, improve the picking efficiency, and effectively reduce the collision between the end executing mechanism and the fruit tree trunk during picking, thereby reducing the collision damage to the fruit tree trunk. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 An execution flow block diagram of the fruit picking auxiliary positioning visual sound and light alarm method of the present application;

[0038] Figure 2 A structural diagram of the picking device in the present application;

[0039] Figure 3 A structural diagram of the picking device in the present application when driving the end executing mechanism to approach the fruit tree trunk;

[0040] Figure 4 A structural diagram of the end executing mechanism in the present application;

[0041] Figure 5 A circuit principle block diagram involved in the present application;

[0042] Figure 6 A structural schematic diagram of the fruit picking auxiliary positioning visual sound and light alarm system of the present application;

[0043] Reference signs:

[0044] Picking device 100;

[0045] Amplitude telescopic arm 1;

[0046] End executing mechanism 2, clamping jaw 21, swing assembly 22;

[0047] Radar ranging alarm device 31, visual image device 32, angle acquisition device 33, display control device 34;

[0048] Cab 4. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the specification of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] Embodiment one

[0051] Please refer to the accompanying Figures 1 to 5 The present application provides a fruit picking auxiliary positioning visual sound and light alarm method, which comprises the following steps:

[0052] Step S1, control the amplitude telescopic arm 1 of the picking device 100 to raise the end effector 2 to the required position, and start the radar ranging alarm device 31, visual image device 32 and angle acquisition device 33 on the end effector 2; wherein the radar ranging alarm device 31 is used to measure the distance between the fruit tree trunk in real time, and generate an alarm when the corresponding distance is reached, to remind the operator to stop operating in time to avoid collision between the end effector 2 and the fruit tree trunk; the visual image device 32 is used to collect the image in front of the end effector 2 in real time, and the visual image device 32 has a centering function, which can provide an alignment mark line for the operator to operate, at the same time, the visual image device 32 can also be used to collect real-time angle information on the angle acquisition device 33; the angle acquisition device 33 is used to collect real-time angle information of the end effector 2, and the angle acquisition device 33 can specifically use a gradienter, an angle sensor, etc.; in the specific implementation of the present application, a display and control device 34 is arranged in the cab 4 of the picking device 100, and the radar ranging alarm device 31, visual image device 32 and angle acquisition device 33 are all electrically connected with the display and control device 34, in the process of specific work, the distance measured by the radar ranging alarm device 31, the image collected by the visual image device 32 and the real-time angle information of the angle acquisition device 33 can be transmitted back to the display and control device 34 for display, so that the operator can control the end effector 2 according to the displayed data content;

[0053] Step S2, according to the alignment mark line provided by the visual image device 32, control the picking device 100 to drive the end effector 2 to align with the fruit tree trunk; wherein the alignment mark line provided by the visual image device 32 can be directly generated by software, which has the same realization principle as the vehicle-mounted reversing image, by forming the alignment mark line on both sides of the picture taken by the visual image device 32, so that the operator only needs to operate the end effector 2 to move to the position where the fruit tree trunk is between the alignment mark lines on both sides of the picture when performing the operation, and preferably to the middle position between the alignment mark lines on both sides of the picture, to achieve better alignment effect;

[0054] Step S3, collecting real-time angle information, adjusting the posture of the end executing mechanism 2 according to the real-time angle information, so that the end executing mechanism 2 can approach the fruit tree trunk in a horizontal posture;

[0055] Step S4, using the radar ranging alarm device 31 to detect the distance between the end executing mechanism 2 and the fruit tree trunk in real time, and triggering corresponding alarms according to the detected distance between the end executing mechanism 2 and the fruit tree trunk, while controlling the picking device 100 to drive the end executing mechanism 2 to move quickly and accurately towards the fruit tree trunk, and the operator can specifically control the end executing mechanism 2 to perform corresponding operations according to the actual alarm information generated; by using the radar ranging alarm device 31 to detect the distance between the end executing mechanism 2 and the fruit tree trunk in real time, the present application can achieve centimeter-level positioning, and once the set distance value is reached, corresponding alarms are triggered, which can help the operator to control the end executing mechanism 2 to quickly locate and clamp the fruit tree trunk.

[0056] By setting the radar ranging alarm device 31, the visual image device 32 and the angle acquisition device 33 on the end executing mechanism 2, and in the specific picking process, first using the alignment mark line provided by the visual image device 32 to align the end executing mechanism 2, that is, the fruit tree trunk is between the alignment mark lines on both sides of the picture; then collecting real-time angle information, and adjusting the posture of the end executing mechanism 2 according to the real-time angle information, so that the end executing mechanism 2 can approach the fruit tree trunk in a horizontal posture; finally, using the radar ranging alarm device 31 to detect the distance between the end executing mechanism 2 and the fruit tree trunk in real time, and triggering corresponding alarms according to the distance; therefore, by using the above technical scheme of the present application, the operator can conveniently operate the end executing mechanism 2 to achieve accurate positioning in the air, improve the picking efficiency, and effectively reduce the collision between the end executing mechanism 2 and the fruit tree trunk during picking, thereby reducing the damage to the fruit tree trunk caused by collision.

[0057] In some embodiments of the present application, the triggering of corresponding alarms according to the distance between the end executing mechanism 2 and the fruit tree trunk is specifically: dividing the alignment mark line into at least three identification areas from one end close to the end executing mechanism 2 to the other end away from the end executing mechanism 2, and pre-setting corresponding radar alarm data for each identification area to generate different alarms according to different identification areas; according to the detected distance between the end executing mechanism 2 and the fruit tree trunk, the identification area where the fruit tree trunk is located is determined, and corresponding alarms are triggered according to the radar alarm data corresponding to the identification area.

[0058] Further, the alarm triggered according to the distance between the end effector 2 and the trunk of the fruit tree is further specifically:

[0059] The alignment mark line is divided into a dangerous identification area, a close identification area and a safe identification area from the end close to the end effector 2 to the end away from the end effector 2, the dangerous identification area is provided with first radar alarm data, the close identification area is provided with second radar alarm data, and the safe identification area is provided with third radar alarm data; for example, the first radar alarm data is set to generate a fast audible and light alarm signal, the second radar alarm data is set to generate a medium-speed audible and light alarm signal, and the third radar alarm data is set to generate a slow-speed audible and light alarm signal;

[0060] When it is determined according to the detected distance between the end effector 2 and the trunk of the fruit tree that the trunk of the fruit tree is in the safe identification area, the corresponding alarm is triggered according to the third radar alarm data, and the picking device 100 is controlled to drive the end effector 2 to move quickly towards the trunk of the fruit tree; because when it is determined that the trunk of the fruit tree is in the safe identification area, it means that the trunk of the fruit tree is far away from the end effector 2, at this time, a slow-speed audible and light alarm signal can be triggered to remind the operator to control the picking device 100 to drive the end effector 2 to quickly approach the trunk of the fruit tree, so as to improve the picking efficiency;

[0061] When it is determined according to the detected distance between the end effector 2 and the trunk of the fruit tree that the trunk of the fruit tree is in the close identification area, the corresponding alarm is triggered according to the second radar alarm data, and the picking device 100 is controlled to drive the end effector 2 to move at a reduced speed towards the trunk of the fruit tree; because when it is determined that the trunk of the fruit tree is in the close identification area, it means that the trunk of the fruit tree is close to the end effector 2, at this time, a medium-speed audible and light alarm signal can be triggered to remind the operator to control the picking device 100 to drive the end effector 2 to approach the trunk of the fruit tree at a reduced speed, so as to avoid collision between the end effector 2 and the trunk of the fruit tree due to too high speed;

[0062] When it is determined according to the detected distance between the end effector 2 and the trunk of the fruit tree that the trunk of the fruit tree is in the dangerous identification area, the corresponding alarm is triggered according to the first radar alarm data, and the picking device 100 is controlled to stop driving the end effector 2 to move towards the trunk of the fruit tree; because when it is determined that the trunk of the fruit tree is in the dangerous identification area, it means that the trunk of the fruit tree has entered the clamping jaw 21 of the end effector 2, at this time, a fast audible and light alarm signal can be triggered to remind the operator to control the end effector 2 to stop moving, so as to prevent collision between the end effector 2 and the trunk of the fruit tree.

[0063] The application sets three areas of danger identification area, proximity identification area and safety identification area, and sets corresponding radar alarm data for each area, so that in the process of specific work, safety, proximity and danger three-stage prompt alarm information can be triggered and generated according to the distance between the end effector 2 and the trunk of the fruit tree, and the operator is fed back, so as to facilitate the operator to perform different control operations according to different alarm information.

[0064] In some embodiments of the application, the collected real-time angle information is specifically:

[0065] The real-time angle information is collected by the angle collection device 33 and is fed back to the cab 4 of the picking device 100 for display, that is, the collected real-time angle information is directly fed back to the display and control device 34 of the cab 4 by the angle collection device 33, and is displayed after being processed by the display and control device 34.

[0066] Or the real-time angle information displayed on the angle collection device 33 is collected by the visual image device 32 and is fed back to the cab 4 of the picking device 100 for display, that is, the angle collection device 33 has a display function and can display real-time angle information, and in work, the real-time angle information on the angle collection device 33 is collected by the visual image device 32 and is fed back to the display and control device 34 of the cab 4 together with the picture collected by the visual image device 32 for display. In the specific implementation of the application, for the feedback of the real-time angle information, any one of the above two ways can be used according to actual needs.

[0067] In some embodiments of the present application, one side of the end effector 2 is provided with a clamping jaw 21 for clamping the trunk of the fruit tree, and the radar ranging alarm device 31 is arranged at the middle position of the top of the end effector 2, so that the radar ranging alarm device 31 can better measure the distance between the end effector 2 and the trunk of the fruit tree; the visual image device 32 is arranged at the middle position of the top of the end effector 2, the visual image device 32 is located away from the clamping jaw 21, and the shooting end of the visual image device 32 faces the opening of the clamping jaw 21; the angle acquisition device 33 is arranged at the top of the end effector 2, when it is needed to acquire the real-time angle information of the angle acquisition device 33 by using the visual image device 32, the angle acquisition device 33 needs to be in the shooting range of the visual image device 32, and the specific position of the angle acquisition device 33 can be arranged according to actual needs, as long as the visual image device 32 can acquire the real-time angle information of the angle acquisition device 33. In the present application, the visual image device 32 is arranged at the middle position of the top of the end effector 2, the shooting end of the visual image device 32 faces the opening of the clamping jaw 21, and the angle acquisition device 33 is also in the shooting range of the visual image device 32, so that in the specific use process, the image of the opening position of the clamping jaw 21 can be acquired by using the visual image device 32, so as to control the end effector 2 to approach the trunk of the fruit tree with a large angle of view, and the real-time angle information of the angle acquisition device 33 can also be acquired by using the visual image device 32 and returned to the display control equipment 34 in the cab 4 for display.

[0068] In the specific implementation of the present application, the swing assembly 22 is arranged between the variable amplitude telescopic arm 1 of the picking device 100 and the middle part of the end effector 2, the swing assembly 22 can specifically include a swing oil cylinder, and the end effector 2 is hinged with the variable amplitude telescopic arm 1, and the swing oil cylinder is connected between the variable amplitude telescopic arm 1 and the end effector 2; in the specific work, the end effector 2 can be driven by the swing assembly 22 to swing relative to the variable amplitude telescopic arm 1, so as to adjust the posture of the end effector 2 to a horizontal state; in the process of swing adjustment of the end effector 2, the radar ranging alarm device 31, the angle acquisition device 33 and the visual image device 32 all swing synchronously with the end effector 2.

[0069] Based on the same inventive concept, the present application also provides a system corresponding to the method in embodiment one, which is described in detail in embodiment two.

[0070] Embodiment two

[0071] In the present embodiment, a fruit picking auxiliary positioning visual sound and light alarm system is provided, please refer to Figures 2 to 6 As shown in the figure, the system comprises a starting module, an alignment module, a posture adjusting module and a motion alarm module.

[0072] The starting module is used for controlling the upwarping of the variable amplitude telescopic arm 1 of the picking device 100 to drive the end effector 2 to rise to a required position, and starting the radar ranging alarm device 31, the visual image device 32 and the angle acquisition device 33 on the end effector 2; wherein the radar ranging alarm device 31 is used for measuring the distance between the end effector 2 and the trunk of the fruit tree in real time, and producing an alarm when reaching a corresponding distance, so as to remind the operator to stop operating in time to avoid the collision between the end effector 2 and the trunk of the fruit tree; the visual image device 32 is used for collecting the image in front of the end effector 2 in real time, and the visual image device 32 has a centering function, which can provide an alignment mark line for the operator to perform centering operation when in use, and the visual image device 32 can also be used for collecting real-time angle information on the angle acquisition device 33; the angle acquisition device 33 is used for collecting real-time angle information of the end effector 2, and the angle acquisition device 33 can specifically adopt a gradienter, an angle sensor or the like; in the specific implementation of the present application, the display and control equipment 34 is arranged in the cab 4 of the picking device 100, and the radar ranging alarm device 31, the visual image device 32 and the angle acquisition device 33 are all electrically connected with the display and control equipment 34, and in the process of specific work, the distance measured by the radar ranging alarm device 31, the image collected by the visual image device 32 and the real-time angle information of the angle acquisition device 33 can be transmitted back to the display and control equipment 34 for display, so as to facilitate the operator to control the end effector 2 according to the displayed data content;

[0073] The alignment module is used for controlling the picking device 100 to drive the end effector 2 to align with the trunk of the fruit tree according to the alignment mark line provided by the visual image device 32; wherein the alignment mark line provided by the visual image device 32 can be directly generated by software, and has the same realization principle as the vehicle-mounted reversing image, which forms the alignment mark line on both sides of the picture taken by the visual image device 32, so that the operator only needs to move the end effector 2 to make the trunk of the fruit tree between the alignment mark lines on both sides of the picture when performing specific operation, and preferably makes the trunk of the fruit tree at the middle position between the alignment mark lines on both sides of the picture, so as to achieve better alignment effect;

[0074] The posture adjusting module is used for collecting real-time angle information, and adjusting the posture of the end effector 2 according to the real-time angle information, so that the end effector 2 can approach the trunk of the fruit tree in a horizontal posture;

[0075] The movement alarm module is used for detecting the distance between the end executing mechanism 2 and the fruit tree trunk in real time by using the radar distance alarm device 31, and triggering corresponding alarms according to the detected distance between the end executing mechanism 2 and the fruit tree trunk, while controlling the picking device 100 to drive the end executing mechanism 2 to move quickly and accurately towards the fruit tree trunk, so that the operator can control the end executing mechanism 2 to perform corresponding operations according to the actual alarm information. The radar distance alarm device 31 is used to detect the distance between the end executing mechanism 2 and the fruit tree trunk in real time, so that the centimeter-level positioning can be well achieved, and once the set distance value is reached, the corresponding alarm is triggered, which can well help the operator to control the end executing mechanism 2 to quickly locate and clamp the fruit tree trunk.

[0076] The radar distance alarm device 31, the visual image device 32 and the angle acquisition device 33 are arranged on the end executing mechanism 2, and in the specific picking process, the end executing mechanism 2 is first aligned by using the alignment mark line provided by the visual image device 32, that is, the fruit tree trunk is located between the alignment mark lines on both sides of the picture; then the real-time angle information is collected, and the posture of the end executing mechanism 2 is adjusted according to the real-time angle information, so that the end executing mechanism 2 can approach the fruit tree trunk in a horizontal posture; finally, the radar distance alarm device 31 is used to detect the distance between the end executing mechanism 2 and the fruit tree trunk in real time, and the corresponding alarm is triggered according to the distance. Therefore, by using the above technical scheme of the present application, the operator can conveniently operate the end executing mechanism 2 to achieve accurate positioning in the air, improve the picking efficiency, and effectively reduce the collision between the end executing mechanism 2 and the fruit tree trunk during picking, thereby reducing the collision damage to the fruit tree trunk.

[0077] In some embodiments of the present application, the triggering of the corresponding alarm according to the distance between the end executing mechanism 2 and the fruit tree trunk is specifically that the alignment mark line is divided into at least three mark areas from one end close to the end executing mechanism 2 to one end away from the end executing mechanism 2, and the corresponding radar alarm data is set for each mark area to generate different alarms according to different mark areas; the mark area where the fruit tree trunk is located is determined according to the detected distance between the end executing mechanism 2 and the fruit tree trunk, and the corresponding alarm is triggered according to the radar alarm data corresponding to the mark area.

[0078] Further, the triggering of the corresponding alarm according to the distance between the end executing mechanism 2 and the fruit tree trunk is further specifically that:

[0079] The alignment mark line is sequentially divided into a dangerous identification area, an approaching identification area and a safe identification area from the end close to the end effector 2 to the end away from the end effector 2, the dangerous identification area is provided with first radar alarm data, the approaching identification area is provided with second radar alarm data, and the safe identification area is provided with third radar alarm data; for example, the first radar alarm data can be set to generate a fast audible and light alarm signal, the second radar alarm data can be set to generate a medium-speed audible and light alarm signal, and the third radar alarm data can be set to generate a slow-speed audible and light alarm signal;

[0080] When it is determined according to the detected distance between the end effector 2 and the fruit tree trunk that the fruit tree trunk is in the safe identification area, the corresponding alarm is triggered according to the third radar alarm data, and the picking device 100 drives the end effector 2 to move quickly towards the fruit tree trunk; because when it is determined that the fruit tree trunk is in the safe identification area, it means that the fruit tree trunk is far away from the end effector 2, at this time, a slow-speed audible and light alarm signal can be triggered to remind the operator to control the picking device 100 to drive the end effector 2 to quickly approach the fruit tree trunk, so as to improve the picking efficiency;

[0081] When it is determined according to the detected distance between the end effector 2 and the fruit tree trunk that the fruit tree trunk is in the approaching identification area, the corresponding alarm is triggered according to the second radar alarm data, and the picking device 100 drives the end effector 2 to move at a reduced speed towards the fruit tree trunk; because when it is determined that the fruit tree trunk is in the approaching identification area, it means that the fruit tree trunk is close to the end effector 2, at this time, a medium-speed audible and light alarm signal can be triggered to remind the operator to control the picking device 100 to drive the end effector 2 to approach the fruit tree trunk at a reduced speed, so as to avoid collision between the end effector 2 and the fruit tree trunk due to too high speed;

[0082] When it is determined according to the detected distance between the end effector 2 and the fruit tree trunk that the fruit tree trunk is in the dangerous identification area, the corresponding alarm is triggered according to the first radar alarm data, and the picking device 100 stops driving the end effector 2 to move towards the fruit tree trunk; because when it is determined that the fruit tree trunk is in the dangerous identification area, it means that the fruit tree trunk has entered the clamping jaw 21 of the end effector 2, at this time, a fast audible and light alarm signal can be triggered to remind the operator to control the end effector 2 to stop moving, so as to prevent collision between the end effector 2 and the fruit tree trunk.

[0083] This invention sets up three zones: a danger zone, a proximity zone, and a safety zone. Each zone is equipped with corresponding radar alarm data. This allows the system to generate three levels of alarm information (safety, proximity, and danger) based on the distance between the end effector 2 and the tree trunk during operation. This information is then fed back to the operator, enabling the operator to perform different control operations based on the different alarm messages.

[0084] In some embodiments of the present invention, the acquisition of real-time angle information specifically includes:

[0085] The angle acquisition device 33 is used to collect real-time angle information and transmits the real-time angle information back to the cab 4 of the harvesting equipment 100 for display. That is, the real-time angle information collected by the angle acquisition device 33 is directly transmitted back to the display and control device 34 in the cab 4, and then processed and displayed by the display and control device 34.

[0086] Alternatively, the visual imaging device 32 can be used to collect real-time angle information displayed on the angle acquisition device 33, and this real-time angle information can be transmitted back to the cab 4 of the harvesting equipment 100 for display. That is, the angle acquisition device 33 has a display function and can display real-time angle information. During operation, the visual imaging device 32 collects real-time angle information from the angle acquisition device 33, and this information, along with the image captured by the visual imaging device 32, is transmitted back to the display and control device 34 in the cab 4 for display. In specific implementations of this invention, either of the above two methods can be used for transmitting real-time angle information according to actual needs.

[0087] In some embodiments of the present invention, the end effector 2 is provided with a gripper 21 for clamping the trunk of a fruit tree on one side. The radar ranging alarm device 31 is located at the middle position of the top of the end effector 2, so that the radar ranging alarm device 31 can better measure the distance between itself and the trunk of the fruit tree. The visual imaging device 32 is located at the middle position of the top of the end effector 2, on the side away from the gripper 21, and the shooting end of the visual imaging device 32 faces the opening of the gripper 21. The angle acquisition device 33 is located at the top of the end effector 2. When it is necessary to use the visual imaging device 32 to collect real-time angle information on the angle acquisition device 33, the angle acquisition device 33 needs to be within the shooting range of the visual imaging device 32. The specific position of the angle acquisition device 33 can be set according to actual needs, as long as the visual imaging device 32 can collect the real-time angle information on the angle acquisition device 33. This invention places the visual imaging device 32 at the middle position of the top of the end effector 2, with the shooting end of the visual imaging device 32 facing the opening of the gripper 21, and the angle acquisition device 33 also within the shooting range of the visual imaging device 32. This allows the visual imaging device 32 to acquire images of the opening position of the gripper 21 during actual use, so as to control the end effector 2 to approach the fruit tree trunk from a wide angle. It also allows the visual imaging device 32 to capture real-time angle information from the angle acquisition device 33 and transmit it back to the display and control device 34 in the cab 4 for display.

[0088] In a specific implementation of this invention, a swing assembly 22 is provided between the midpoint of the luffing telescopic arm 1 and the end effector 2 of the harvesting device 100. The swing assembly 22 may specifically include a swing cylinder, and the end effector 2 is hinged to the luffing telescopic arm 1. The swing cylinder is connected between the luffing telescopic arm 1 and the end effector 2. In actual operation, the swing assembly 22 can be used to drive the end effector 2 to swing relative to the luffing telescopic arm 1, thereby adjusting the attitude of the end effector 2 to a horizontal state. During the swing adjustment of the end effector 2, the radar ranging alarm device 31, the angle acquisition device 33, and the visual imaging device 32 will all swing synchronously with the end effector 2.

[0089] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A fruit picking auxiliary positioning visual and acoustic alarm method, characterized in that, The method comprises the following steps: controlling the amplitude telescopic arm of the picking device to raise the end effector to a required position, and starting the radar ranging alarm device, visual image device and angle acquisition device on the end effector; controlling the picking device to drive the end effector to align with the trunk of the fruit tree according to the alignment mark line provided by the visual image device; acquiring real-time angle information, and adjusting the posture of the end effector according to the real-time angle information so that the end effector can approach the trunk of the fruit tree in a horizontal posture; detecting the distance between the end effector and the trunk of the fruit tree in real time by using the radar ranging alarm device, and triggering corresponding alarms according to the detected distance between the end effector and the trunk of the fruit tree, while controlling the picking device to drive the end effector to move quickly and accurately towards the trunk of the fruit tree.

2. The fruit picking auxiliary positioning visual and acoustic alarm method according to claim 1, characterized in that, The specific method of triggering corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from one end close to the end effector to the other end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area.

3. The fruit picking auxiliary positioning visual and acoustic alarm method according to claim 2, characterized in that, The specific method of triggering corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from one end close to the end effector to the other end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. The specific method of triggering corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from one end close to the end effector to the other end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. The specific method of triggering corresponding alarms according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from one end close to the end effector to the other end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. The specific method of acquiring real-time angle information is that: real-time angle information is acquired by using the angle acquisition device, and the real-time angle information is transmitted back to the cab of the picking device for display; 4. The fruit picking auxiliary positioning visual and acoustic alarm method according to claim 1, characterized in that, or the real-time angle information displayed on the angle acquisition device is acquired by using the visual image device, and the real-time angle information is transmitted back to the cab of the picking device for display. ​ ​ 5. The fruit picking auxiliary positioning visual and acoustic alarm method according to claim 1, characterized in that, The end effector is provided with a clamping jaw on one side for clamping the trunk of the fruit tree, a radar ranging alarm device is arranged at the middle position of the top of the end effector, a visual image device is arranged at the middle position of the top of the end effector, the visual image device is located on the side away from the clamping jaw, and the shooting end of the visual image device faces the opening of the clamping jaw, and an angle acquisition device is arranged on the top of the end effector.

6. A fruit picking auxiliary positioning visual and acoustic alarm system, characterized in that, The system comprises a starting module, an alignment module, a posture adjustment module and a motion alarm module; The starting module is used for controlling the variable amplitude telescopic arm of the picking device to drive the end effector to ascend to a required position, and starting the radar ranging alarm device, the visual image device and the angle acquisition device on the end effector; The alignment module is used for controlling the picking device to drive the end effector to align with the trunk of the fruit tree according to the alignment mark line provided by the visual image device; The posture adjustment module is used for collecting real-time angle information, and adjusting the posture of the end effector according to the real-time angle information, so that the end effector can approach the trunk of the fruit tree in a horizontal posture; The motion alarm module is used for detecting the distance between the end effector and the trunk of the fruit tree in real time by using the radar ranging alarm device, and triggering corresponding alarms according to the detected distance between the end effector and the trunk of the fruit tree, and simultaneously controlling the picking device to drive the end effector to move quickly and accurately towards the trunk of the fruit tree.

7. The fruit picking auxiliary positioning visual and acoustic alarm system according to claim 6, characterized in that, The specific alarm triggered according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from the end close to the end effector to the end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area.

8. The fruit picking auxiliary positioning visual and acoustic alarm system according to claim 7, characterized in that, The specific alarm triggered according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from the end close to the end effector to the end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. The specific alarm triggered according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from the end close to the end effector to the end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. The specific alarm triggered according to the distance between the end effector and the trunk of the fruit tree is that the alignment mark line is divided into at least three mark areas from the end close to the end effector to the end away from the end effector, and corresponding radar alarm data is set for each mark area; the mark area where the trunk of the fruit tree is located is determined according to the detected distance between the end effector and the trunk of the fruit tree, and corresponding alarms are triggered according to the radar alarm data corresponding to the mark area. ​ When it is determined that the fruit tree trunk is in the dangerous identification area according to the distance between the detected end effector and the fruit tree trunk, a corresponding alarm is triggered according to the first radar alarm data, and the picking device is controlled to stop driving the end effector to move towards the fruit tree trunk.

9. The fruit picking auxiliary positioning visual and acoustic alarm system according to claim 6, characterized in that, The real-time angle information is collected by an angle collection device, and the real-time angle information is transmitted back to the cab of the picking device for display. The real-time angle information is collected by an angle collection device, and the real-time angle information is transmitted back to the cab of the picking device for display. The real-time angle information is collected by an angle collection device, and the real-time angle information is transmitted back to the cab of the picking device for display.

10. The fruit picking auxiliary positioning visual and acoustic alarm system according to claim 6, characterized in that, One side of the end effector is provided with a clamping jaw for clamping the fruit tree trunk, the radar ranging alarm device is arranged at the middle position of the top of the end effector, the visual image device is arranged at the middle position of the top of the end effector, the visual image device is located away from the clamping jaw, and the shooting end of the visual image device faces the opening of the clamping jaw, and the angle collection device is arranged at the top of the end effector.

Images