Apple picking robot intelligent path planning system and method based on deep learning

Abstract

The invention discloses an apple picking robot intelligent path planning system and method based on deep learning, and relates to the field of robot path planning, and the method comprises the steps: obtaining the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in a picking region; picking apples on the two sides of the apple tree by adopting a complementary apple picking mode; obtaining a dynamic delay coefficient prediction value of apple picking in a single-frame area of the picking robots on the two sides; dynamic picking adaptability functions of the picking robots on the two sides are obtained; determining an implementation principle of dynamic picking path planning of the picking robot; an intelligent path planning model is established, and the optimal path planning of the apple picking robot is obtained; a man-machine interaction platform is established, and monitoring data of the apple picking robot in the picking process are stored, displayed and warned in an early warning mode. The picking efficiency of the apple picking robot is effectively improved, the light influence is reduced, and the flexibility of the picking robot is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of robot path planning, and more particularly to an intelligent path planning system and method for an apple picking robot based on deep learning. Background Art

[0002] Apple picking robots can continuously pick ripe apples. Compared with traditional manual picking, they not only greatly improve the efficiency of apple picking, but also save a large amount of labor costs. They are the core driving force for the transformation of agriculture from labor-intensive to intelligent. Planning the optimal picking path for apple picking robots is an important means to improve the picking efficiency and the ability to handle abnormal events of apple picking robots. Therefore, it is of great significance to improve the intelligent path planning of apple picking robots.

[0003] Existing apple picking robots use two-sided picking or one-sided picking. However, two-sided picking is extremely complex compared to one-sided picking, with high requirements for the picking environment and the picking robot, slow system processing speed, and prone to missed picking and slow efficiency during the picking process. Moreover, due to the change of light, when the apple picking robot collects on both sides of the apple tree, it is easily affected by the back-and-forth switching between the sunny side and the shady side, resulting in an increase in the shadow area when the robot obtains pictures, poor quality of the collected pictures, and inaccurate detection during apple picking. Summary of the Invention

[0004] To solve the above technical problems, an intelligent path planning system and method for an apple picking robot based on deep learning are provided. This technical solution solves the problems mentioned in the above background art, such as the high requirements for the picking environment and the picking robot in two-sided picking, slow system processing speed, prone to missed picking and slow efficiency during the picking process. Moreover, due to the change of light, when the apple picking robot collects on both sides of the apple tree, it is easily affected by the back-and-forth switching between the sunny side and the shady side, resulting in an increase in the shadow area when the robot obtains pictures, poor quality of the collected pictures, and inaccurate detection during apple picking.

[0005] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] An intelligent path planning method for an apple picking robot based on deep learning, comprising:

[0007] According to the apple planting distribution, obtain the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area;

[0008] Adopt a complementary apple picking method to pick the apples on both sides of the apple tree, and improve the apple picking efficiency and reduce the influence of light through dual-machine cooperation;

[0009] Obtain the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the apple picking robots on both sides according to the picking efficiency of the apple picking robots.

[0010] Obtain the adaptive function for dynamic picking of the apple picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process.

[0011] Determine the implementation principle of the dynamic picking path planning of the apple picking robots according to the adaptive function for dynamic picking of the apple picking robots on both sides.

[0012] Based on the deep learning algorithm model, combine the predicted value of the dynamic delay coefficient and the adaptive function for dynamic picking to establish an intelligent path planning model, and obtain the optimal path planning of the apple picking robots.

[0013] Establish a human-machine interaction platform for storing, displaying and warning the monitoring data of the apple picking robots during the picking process, as well as the initial path planning.

[0014] Preferably, the obtaining of the channel spacing between adjacent two hedges, the length of each hedge and the number of hedges in the picking area according to the apple planting distribution situation specifically includes:

[0015] Obtain the channel spacing between adjacent two hedges, the length of each hedge and the number of hedges in the picking area according to the apple planting distribution in the picking area.

[0016] Number each hedge according to the number of apple planting hedges in the picking area, and establish an apple planting list for the picking area.

[0017] Among them, the hedge numbers in the apple planting list for the picking area correspond one by one to the data of the channel spacing between adjacent two hedges, the length of each hedge and the number of hedges in the picking area.

[0018] Preferably, the adoption of the complementary apple picking method to pick the apples on both sides of the apple tree and improve the apple picking efficiency and reduce the influence of light through dual-machine cooperation specifically includes:

[0019] The adoption of the complementary apple picking method specifically includes:

[0020] Set two robots as a group. At the initial stage of apple picking, the two robots are respectively located on both sides of the apple tree and drive towards each other.

[0021] The robot on the left side of the apple tree gives priority to picking the apples on the left side of the apple tree, and the robot on the right side of the apple tree gives priority to picking the apples on the right side of the apple tree.

[0022] Optimize the image quality collected by the picking robots on both sides according to the influence of unidirectional light, and reduce the influence of unidirectional light on the picking robots.

[0023] Preferably, the specific steps of obtaining the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the picking robots on both sides according to the picking efficiency of the apple picking robots are as follows:

[0024] Obtain the number of apples in the single-frame collection area of the picking robot according to the area range of the apple tree collected by the apple picking robot;

[0025] According to the set apple picking efficiency of the apple picking robot and combined with the number of apples in the single-frame collection area, obtain the effective time for the picking robot to complete the apple picking in the single-frame collection area;

[0026] Obtain the relative value of the dynamic delay for the picking robots on both sides to complete the apple picking in the single-frame collection area according to the effective time for the picking robots on both sides to complete the apple picking in the single-frame collection area;

[0027] Determine the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the picking robots on both sides according to the relative value of the dynamic delay for the picking robots on both sides to complete the apple picking in the single-frame collection area;

[0028] Establish an input matrix of the predicted value of the dynamic delay coefficient according to the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the picking robots on both sides;

[0029] The expression for obtaining the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the picking robots on both sides is:

[0030]

[0031] In the formula, A i and B i are respectively the effective times for the left and right picking robots to complete the apple picking in the i-th single-frame collection area, are respectively the numbers of apples in the single-frame collection areas of the left and right picking robots, γ is the picking efficiency of the picking robot, T i is the relative value of the dynamic delay for the picking robots on both sides to complete the apple picking in the i-th single-frame collection area, is the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the picking robot, δ is the balance constant term, and N is the number of apples picked by the picking robots on both sides in the single-frame collection area.

[0032] Preferably, the specific steps of obtaining the adaptive function for dynamic picking by the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process are as follows:

[0033] Obtain the number of units for fixed apple picking for each hedge according to the length of each hedge and the distance of the single-frame acquisition area of the picking robot.

[0034] Set the position acquisition devices of the picking robots on both sides to obtain the position information of the picking robots on both sides.

[0035] Obtain the displacement time of the picking robots on both sides according to the position information of the picking robots on both sides and the fixed moving speed of the robots.

[0036] Obtain the relative time for danger avoidance and the relative time for collision waiting of the picking robots on both sides according to the change situation of the position information of the picking robots on both sides.

[0037] Obtain the adaptive function of dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process.

[0038] The adaptive function of dynamic picking of the picking robots on both sides is as follows:

[0039]

[0040] In the formula, f is the value of the adaptive function of dynamic picking of the picking robots on both sides, and Δt i is the relative displacement time for the picking robots on both sides to complete the apple picking in the i-th single-frame acquisition area. α is the weight of the relative time for collision waiting of the picking robot, and ΔT z is the relative time for collision waiting of the picking robot. β is the weight of the relative time for danger avoidance of the picking robot, and ΔT s is the relative time for danger avoidance of the picking robot.

[0041] Preferably, the implementation principles for determining the dynamic picking path planning of the picking robots according to the adaptive function of dynamic picking of the picking robots on both sides specifically include:

[0042] Based on big data, set the phased threshold for channel occupancy between two adjacent hedges.

[0043] The phased threshold for channel occupancy between two adjacent hedges includes three stages: a small threshold, a medium threshold, and a large threshold. Among them, the range of the small threshold is: [0, θ1τ], the range of the medium threshold is: (θ1τ, θ2τ], and the range of the large threshold is: (θ2τ, ∞). Here, τ is the total time spent by the first picking robot to complete the apple picking task of a single hedge, θ1 is the equilibrium multiple coefficient of the small threshold, and θ2 is the equilibrium multiple coefficient of the medium threshold.

[0044] Determine the implementation principles for the dynamic picking path planning of the picking robots according to the adaptive function of dynamic picking of the picking robots on both sides.

[0045] According to the implementation principles of the dynamic picking path planning of the picking robot, guide the apple picking methods of the picking robots on both sides.

[0046] Preferably, based on the deep learning algorithm model, combined with the dynamic delay coefficient prediction value and the dynamic picking adaptability function, establish an intelligent path planning model to obtain the optimal path planning of the apple picking robot, which specifically includes:

[0047] Extract the feature items of the intelligent path planning of the apple picking robot according to the dynamic delay coefficient prediction value, the adaptability function and the implementation principles of path planning;

[0048] Based on big data or test experiments, obtain the training sample set and the target sample set of the intelligent path planning of the apple picking robot;

[0049] Based on the deep learning algorithm model, establish an intelligent path planning model, and obtain the optimal path planning of the apple picking robot by real-time obtaining the dynamic delay coefficient prediction values and the dynamic picking adaptability functions of the picking robots on both sides, as well as the judgment situation of the implementation principles of the dynamic picking path planning of the picking robot.

[0050] Preferably, the establishment of the human-computer interaction platform for storing, displaying and warning the monitoring data of the apple picking robot during the picking process, as well as the initial path planning, specifically includes:

[0051] Establish a human-computer interaction platform, and based on the Internet of Things technology, receive and store the monitoring data of the picking robots at both ends during the picking process;

[0052] Based on the human-computer interaction platform, build the operating environment of the intelligent path planning model, and update and optimize the practicability of the model by optimizing parameters;

[0053] Based on the human-computer interaction platform, display and warn the position information and abnormal conditions of the picking robots at both ends during the apple picking process;

[0054] Based on the human-computer interaction platform, complete the initial path planning and abnormal manual remote control of the picking robots at both ends through the human-computer interaction interface.

[0055] Furthermore, this solution proposes an intelligent path planning system for apple picking robots based on deep learning, which is used to implement the intelligent path planning method for apple picking robots based on deep learning as described above, and includes:

[0056] The picking distribution acquisition module, which is used to obtain the channel spacing between adjacent two hedges, the length of each hedge and the number of hedges in the picking area according to the apple planting distribution situation;

[0057] The picking method module is used to pick apples on both sides of the apple tree by using a complementary apple picking method, improving the apple picking efficiency and reducing the influence of light through dual-machine cooperation;

[0058] The path planning module is used to obtain the predicted value of the dynamic delay coefficient for apple picking in a single-frame area of the picking robots on both sides according to the picking efficiency of the apple picking robot; obtain the adaptive function for dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process; determine the implementation principle of the dynamic picking path planning of the picking robot according to the adaptive function for dynamic picking of the picking robots on both sides; establish an intelligent path planning model based on the deep learning algorithm model, combining the predicted value of the dynamic delay coefficient and the adaptive function for dynamic picking, and obtain the optimal path planning of the apple picking robot;

[0059] The interactive platform module is used to establish a human-machine interactive platform for storing, displaying and warning the monitoring data of the apple picking robot during the picking process, as well as the initial path planning.

[0060] Preferably, the path planning module includes:

[0061] The dynamic delay unit is used to obtain the predicted value of the dynamic delay coefficient for apple picking in a single-frame area of the picking robots on both sides according to the picking efficiency of the apple picking robot;

[0062] The adaptive function unit is used to obtain the adaptive function for dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process;

[0063] The implementation principle unit is used to determine the implementation principle of the dynamic picking path planning of the picking robot according to the adaptive function for dynamic picking of the picking robots on both sides;

[0064] The path planning unit is used to establish an intelligent path planning model based on the deep learning algorithm model, combining the predicted value of the dynamic delay coefficient and the adaptive function for dynamic picking, and obtain the optimal path planning of the apple picking robot.

[0065] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0066] By adopting a complementary apple picking method, apples on both sides of the apple tree are picked. Through the cooperation of two machines, the apple picking efficiency is improved and the influence of light is reduced. Secondly, according to the area range of the apple tree collected by the apple picking robot, the number of apples in the single-frame collection area of the picking robot is obtained. And according to the picking efficiency of the apple picking robot, the predicted value of the dynamic delay coefficient of apple picking in the single-frame area of the picking robots on both sides is obtained. Then, according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process, the adaptive function of the dynamic picking of the picking robots on both sides is obtained. Furthermore, by judging the range of the adaptive function of the dynamic picking of the picking robots on both sides and the phased threshold of the channel occupancy between two adjacent hedges, the apple picking method of the picking robots on both sides is guided, so as to determine the implementation principle of the dynamic picking path planning of the picking robot. Finally, based on the deep learning algorithm model, an intelligent path planning model is established. By obtaining the predicted value of the dynamic delay coefficient and the dynamic picking adaptive function of the picking robots on both sides in real time, as well as the judgment situation of the implementation principle of the dynamic picking path planning of the picking robot, the optimal path planning of the apple picking robot is obtained, thus effectively improving the picking efficiency of the apple picking robot and reducing the influence of light, and greatly increasing the flexibility of the picking robot. Description of the Drawings

[0067] Figure 1 Flowchart of an intelligent path planning method for an apple picking robot based on deep learning according to the present invention;

[0068] Figure 2 Flowchart of obtaining the predicted value of the dynamic delay coefficient of apple picking in the single-frame area of the picking robots on both sides according to the picking efficiency of the apple picking robot according to the present invention;

[0069] Figure 3 Flowchart of obtaining the adaptive function of the dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process according to the present invention;

[0070] Figure 4 Flowchart of determining the implementation principle of the dynamic picking path planning of the picking robot according to the adaptive function of the dynamic picking of the picking robots on both sides according to the present invention. Detailed Embodiments

[0071] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments in the following description are only examples, and those skilled in the art can think of other obvious variations.

[0072] Refer to Figure 1 As shown, an intelligent path planning method for an apple picking robot based on deep learning includes:

[0073] According to the apple planting distribution, obtain the passage spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area;

[0074] Adopt a complementary apple picking method to pick the apples on both sides of the apple tree, and improve the apple picking efficiency and reduce the light influence through the cooperation of two machines;

[0075] According to the picking efficiency of the apple picking robot, obtain the predicted value of the dynamic delay coefficient of apple picking in the single-frame area of the picking robots on both sides;

[0076] According to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process, obtain the adaptive function of dynamic picking of the picking robots on both sides;

[0077] According to the adaptive function of dynamic picking of the picking robots on both sides, determine the implementation principle of the dynamic picking path planning of the picking robot;

[0078] Based on the deep learning algorithm model, combine the predicted value of the dynamic delay coefficient and the adaptive function of dynamic picking to establish an intelligent path planning model, and obtain the optimal path planning of the apple picking robot;

[0079] Establish a human-machine interaction platform for storing, displaying and warning the monitoring data of the apple picking robot during the picking process, as well as the initial path planning.

[0080] It can be explained that in this solution, by adopting a complementary apple picking method, the apples on both sides of the apple tree are picked, and the apple picking efficiency is improved and the light influence is reduced through the cooperation of two machines. Secondly, according to the area range of the apple tree collected by the apple picking robot, the number of apples in the single-frame collection area of the picking robot is obtained, and according to the picking efficiency of the apple picking robot, the predicted value of the dynamic delay coefficient of apple picking in the single-frame area of the picking robots on both sides is obtained, and according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process, the adaptive function of dynamic picking of the picking robots on both sides is obtained. Furthermore, by judging the range of the adaptive function of dynamic picking of the picking robots on both sides and the stage threshold of the passage occupancy between two adjacent hedges, the apple picking method of the picking robots on both sides is guided, so as to determine the implementation principle of the dynamic picking path planning of the picking robot. Finally, based on the deep learning algorithm model, an intelligent path planning model is established. By real-time obtaining the predicted value of the dynamic delay coefficient and the adaptive function of dynamic picking of the picking robots on both sides, as well as the judgment situation of the implementation principle of the dynamic picking path planning of the picking robot, the optimal path planning of the apple picking robot is obtained, so as to effectively improve the picking efficiency of the apple picking robot and reduce the light influence, and greatly increase the flexibility of the picking robot.

[0081] Refer to Figure 2As shown, obtaining the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the two-sided picking robots according to the picking efficiency of the apple picking robots specifically includes:

[0082] According to the area range of the apple tree collected by the apple picking robot, obtain the number of apples in the single-frame collection area of the picking robot;

[0083] According to the apple picking efficiency set by the apple picking robot, combined with the number of apples in the single-frame collection area, obtain the effective time for the picking robot to complete apple picking in the single-frame collection area;

[0084] According to the effective time for the two-sided picking robots to complete apple picking in the single-frame collection area, obtain the relative value of the dynamic delay for the two-sided picking robots to complete apple picking in the single-frame collection area;

[0085] According to the relative value of the dynamic delay for the two-sided picking robots to complete apple picking in the single-frame collection area, determine the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the two-sided picking robots;

[0086] According to the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the two-sided picking robots, establish an input matrix of the predicted value of the dynamic delay coefficient;

[0087] The expression for obtaining the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the two-sided picking robots is:

[0088]

[0089] In the formula, A i and B i are respectively the effective times for the left and right picking robots to complete apple picking in the i-th single-frame collection area, are respectively the numbers of apples in the single-frame collection areas of the left and right picking robots, γ is the picking efficiency of the picking robot, T i is the relative value of the dynamic delay for the two-sided picking robots to complete apple picking in the i-th single-frame collection area, is the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the picking robot, δ is the balance constant term, and N is the number of the two-sided picking robots to complete apple picking in the single-frame collection area.

[0090] It can be explained that the best result of the complementary apple picking method is that the two side picking robots complete the apple picking on both sides of the apple tree simultaneously, thus ensuring that the apple picking robots do not interfere with each other during apple picking, thereby improving the stability and efficiency of the apple picking robots. However, since the apple data in the single-frame acquisition area is not unique when the apple picking robot is picking apples, and the apple picking time in the single-frame acquisition area is also different under the same picking efficiency, it is necessary to relatively evaluate the apple picking time in the single-frame acquisition area of the two side picking robots. This solution effectively reflects the relative time difference in completing the apple picking in the single-frame acquisition area of the two side picking robots by establishing a predicted value of the dynamic delay coefficient for apple picking in the single-frame area of the two side picking robots, thereby providing data support for subsequent analysis.

[0091] Referring to Figure 3 As shown, the specific steps for obtaining the adaptive function of the dynamic picking of the two side picking robots according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process are as follows:

[0092] Obtain the number of units that need to fixedly pick apples for each hedge according to the length of each hedge and the distance of the single-frame acquisition area of the picking robot;

[0093] Set the position acquisition devices of the two side picking robots to obtain the position information of the two side picking robots;

[0094] Obtain the displacement time of the two side picking robots according to the position information of the two side picking robots and the fixed moving speed of the robots;

[0095] Obtain the relative time for danger avoidance and the relative time for collision waiting of the two side picking robots according to the change situation of the position information of the two side picking robots;

[0096] Obtain the adaptive function of the dynamic picking of the two side picking robots according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process;

[0097] The adaptive function of the dynamic picking of the two side picking robots is:

[0098]

[0099] In the formula, f is the value of the adaptive function of the dynamic picking of the two side picking robots, Δt i is the relative displacement time for the two side picking robots to complete the apple picking in the i-th single-frame acquisition area, α is the weight of the relative time for collision waiting of the picking robot, ΔT z is the relative time for collision waiting of the picking robot, β is the weight of the relative time for danger avoidance of the picking robot, ΔT s is the relative time for danger avoidance of the picking robot.

[0100] It can be explained that when the channel spacing between two adjacent hedges cannot accommodate two picking robots, during the dynamic picking of the picking robots on both sides, they may be affected by environmental obstacles and each other, resulting in a lag in the picking time. As a result, it may lead to a situation where the picking robots on both sides coexist in one channel, thus causing the picking robots to be unable to work. Therefore, to ensure that the picking robots on both sides stagger the same channel during dynamic picking, it is necessary to know the relative time relationship of the dynamic picking of the picking robots on both sides. In this solution, an adaptation function for the dynamic picking of the picking robots on both sides is established, and the relative time relationship of the dynamic picking of the picking robots on both sides is effectively evaluated by using the adaptation function for the dynamic picking of the picking robots on both sides, so as to reduce the influence of environmental obstacles and each other, and improve the stability and orderliness of the dynamic picking of the picking robots on both sides. Among them,

[0101] The specific determination method for obtaining the relative time of danger avoidance and the relative time of collision waiting of the picking robots on both sides according to the change of the position information of the picking robots on both sides is as follows:

[0102] When the distance value between the picking robots on both sides is less than the set value and one position changes while the other remains unchanged, it means that the picking robots on both sides are in collision waiting;

[0103] When the distance value between the picking robots on both sides is greater than the set value and one of the robots remains in place or moves in the opposite direction, it means that the robot is in danger avoidance;

[0104] The expression of the distance value between the picking robots on both sides is: In the formula, s is the distance value between the picking robots on both sides, (x1, y1) is the position coordinates of the left robot, and (x1, y1) is the position coordinates of the right robot.

[0105] Referring to Figure 4 As shown, the implementation principles for determining the dynamic picking path planning of the picking robots according to the adaptation function of the dynamic picking of the picking robots on both sides specifically include:

[0106] Based on big data, set the phased threshold for the channel occupancy between two adjacent hedges;

[0107] The phased threshold for the channel occupancy between two adjacent hedges includes three stages: a small threshold, a medium threshold, and a large threshold. Among them, the small threshold range is: [0, θ1τ], the medium threshold range is: (θ1τ, θ2τ], and the large threshold range is: (θ2τ, ∞), where τ is the total time spent by the first picking robot to complete the apple picking task of a single hedge, θ1 is the equilibrium multiple coefficient of the small threshold, and θ2 is the equilibrium multiple coefficient of the medium threshold;

[0108] Determine the implementation principles of the dynamic picking path planning for the picking robot according to the adaptability function of the dynamic picking of the picking robots on both sides;

[0109] Guide the apple picking methods of the picking robots on both sides according to the implementation principles of the dynamic picking path planning for the picking robot.

[0110] It can be explained that the specific steps of determining the implementation principles of the dynamic picking path planning for the picking robot are as follows:

[0111] Step 1, judge whether the adaptability function value f of the dynamic picking of the picking robots on both sides is greater than θ1τ. If it is, guide the picking robot that has normally completed the apple picking task of a single hedge to pick apples from the i-th hedge to the (i + 2)-th hedge. If not, wait after completing the apple picking task of a single hedge. When the specified time is reached or the other robot returns to normal, go to the (i + 1)-th hedge to pick apples;

[0112] Step 2, judge whether the adaptability function value f of the dynamic picking of the picking robots on both sides is greater than θ2τ. If it is, guide the picking robot that has normally completed all the apple picking tasks of the hedges on one side of the collection area to pick the apples on the other side nearby until the apple picking on the other side is completed.

[0113] Furthermore, based on the same inventive concept as the above-mentioned intelligent path planning method for the apple picking robot based on deep learning, this solution proposes an intelligent path planning system for the apple picking robot based on deep learning, including:

[0114] A picking distribution acquisition module, which is used to obtain the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area according to the apple planting distribution;

[0115] A picking method module, which is used to adopt a complementary apple picking method to pick the apples on both sides of the apple tree, and improve the apple picking efficiency and reduce the influence of light through double-robot cooperation;

[0116] A path planning module, which is used to obtain the predicted value of the dynamic delay coefficient of the apple picking of the picking robots on both sides in a single-frame area according to the picking efficiency of the apple picking robot; obtain the adaptability function of the dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process; determine the implementation principles of the dynamic picking path planning for the picking robot according to the adaptability function of the dynamic picking of the picking robots on both sides; establish an intelligent path planning model based on the deep learning algorithm model, combined with the predicted value of the dynamic delay coefficient and the dynamic picking adaptability function, and obtain the optimal path planning of the apple picking robot;

[0117] An interactive platform module, which is used to establish a human-machine interactive platform for storing, displaying, and warning the monitoring data during the picking process of the apple picking robot, as well as the initial path planning.

[0118] The path planning module includes:

[0119] A dynamic delay unit, which is used to obtain the predicted value of the dynamic delay coefficient for apple picking in the single-frame area of the picking robots on both sides according to the picking efficiency of the apple picking robot.

[0120] An adaptability function unit, which is used to obtain the adaptability function for dynamic picking of the picking robots on both sides according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process.

[0121] An implementation principle unit, which is used to determine the implementation principle of the dynamic picking path planning of the picking robot according to the adaptability function for dynamic picking of the picking robots on both sides.

[0122] A path planning unit, which is used to establish an intelligent path planning model based on a deep learning algorithm model, combined with the predicted value of the dynamic delay coefficient and the adaptability function for dynamic picking, to obtain the optimal path planning of the apple picking robot.

[0123] In summary, the advantages of the present invention are as follows: effectively improving the picking efficiency of the apple picking robot and reducing the influence of light, and greatly increasing the flexibility of the picking robot.

[0124] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification is only the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent path planning method for an apple picking robot based on deep learning, characterized in that, Including: According to the apple planting distribution, obtain the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area; Adopt a complementary apple picking method to pick the apples on both sides of the apple tree, and improve the apple picking efficiency and reduce the light influence through dual-machine cooperation; According to the picking efficiency of the apple picking robot, obtain the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the two-side picking robots; According to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process, obtain the adaptive function for dynamic picking by the two-side picking robots; According to the adaptive function for dynamic picking by the two-side picking robots, determine the implementation principle for the dynamic picking path planning of the picking robot; Based on the deep learning algorithm model, combine the predicted value of the dynamic delay coefficient and the dynamic picking adaptive function to establish an intelligent path planning model, and obtain the optimal path planning for the apple picking robot; Establish a human-machine interaction platform for storing, displaying, and warning the monitoring data of the apple picking robot during the picking process, as well as the initial path planning.

2. The intelligent path planning method for an apple picking robot based on deep learning according to claim 1, wherein The specific steps for obtaining the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area according to the apple planting distribution include: According to the apple planting distribution in the picking area, obtain the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area; Number each hedge according to the number of apple-planting hedges in the picking area, and establish an apple-planting list for the picking area; Among them, the hedge numbers in the apple-planting list for the picking area are in one-to-one correspondence with the data of the channel spacing between two adjacent hedges, the length of each hedge, and the number of hedges in the picking area.

3. The intelligent path planning method of an apple picking robot based on deep learning according to claim 2, characterized in that, The specific steps for adopting a complementary apple picking method to pick the apples on both sides of the apple tree and improving the apple picking efficiency and reducing the light influence through dual-machine cooperation include: The specific steps for adopting a complementary apple picking method include: Set two robots as a group. At the initial stage of apple picking, the two robots are located on both sides of the apple tree and drive towards each other; The robot on the left side of the apple tree gives priority to picking the apples on the left side of the apple tree, and the robot on the right side of the apple tree gives priority to picking the apples on the right side of the apple tree; According to the influence of unidirectional light, optimize the image quality collected by the two-side picking robots respectively to reduce the unidirectional light influence on the picking robots.

4. A method for intelligent path planning of an apple picking robot based on deep learning according to claim 3, characterized in that, The specific steps for obtaining the predicted value of the dynamic delay coefficient for apple picking in a single-frame area by the two-side picking robots according to the picking efficiency of the apple picking robot include: According to the area range of the apple tree collected by the apple picking robot, obtain the number of apples in the single-frame collection area of the picking robot; According to the set apple picking efficiency of the apple picking robot, combine the number of apples in the single-frame collection area to obtain the effective time for the picking robot to complete the apple picking in the single-frame collection area; According to the effective time for the two-side picking robots to complete the apple picking in the single-frame collection area, obtain the relative value of the dynamic delay for the two-side picking robots to complete the apple picking in the single-frame collection area; Determine the predicted value of the dynamic delay coefficient for apple picking in the single-frame acquisition area by the two-sided picking robots based on the relative value of the dynamic delay for apple picking in the single-frame acquisition area by the two-sided picking robots; Establish an input matrix of the predicted value of the dynamic delay coefficient based on the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the two-sided picking robots; The expression for obtaining the predicted value of the dynamic delay coefficient for apple picking in the single-frame area by the two-sided picking robots is: Wherein, A i , B i are respectively the effective times for the left and right picking robots to complete the apple picking in the i-th single-frame acquisition area, are respectively the numbers of apples in the single-frame acquisition areas of the left and right picking robots, γ is the picking efficiency of the picking robot, T i is the relative dynamic delay for the two picking robots to complete the apple picking in the i-th single-frame acquisition area, φ(T) is the predicted value of the dynamic delay coefficient for the apple picking in the single-frame area of the picking robot, δ is the balance constant term, and N is the number of apples picked by the two picking robots in the single-frame acquisition area.

5. The intelligent path planning method for an apple picking robot based on deep learning according to claim 4, characterized in that, The specific steps for obtaining the adaptive function for dynamic picking of the two-sided picking robots according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process include: Obtain the number of units for fixed apple picking for each hedge according to the length of each hedge and the distance of the single-frame acquisition area of the picking robot; Set the position acquisition devices of the two-sided picking robots to obtain the position information of the two-sided picking robots; Obtain the displacement time of the two-sided picking robots according to the position information of the two-sided picking robots and the fixed moving speed of the robots; Obtain the relative time for danger avoidance and the relative time for collision waiting of the two-sided picking robots according to the change situation of the position information of the two-sided picking robots; Obtain the adaptive function for dynamic picking of the two-sided picking robots according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process; The adaptive function for dynamic picking of the two-sided picking robots is: where f is the adaptability function value of dynamic picking by the picking robots on both sides, and Δt i is the relative displacement time for the picking robots on both sides to complete the picking of apples in the i-th single-frame acquisition area, α is the weight of the relative time of waiting for collision of the picking robot, and ΔT z is the relative time of waiting for collision of the picking robot, β is the weight of the relative time of danger avoidance of the picking robot, and ΔT s is the relative time of danger avoidance of the picking robot.

6. The intelligent path planning method of an apple picking robot based on deep learning according to claim 5, characterized in that, The specific steps for determining the implementation principle of the dynamic picking path planning of the picking robots according to the adaptive function for dynamic picking of the two-sided picking robots include: Based on big data, set the phased threshold for channel occupancy between two adjacent hedges; The phased threshold for channel occupancy between two adjacent hedges includes three stages: a small threshold, a medium threshold, and a large threshold. Among them, the range of the small threshold is: [0, θ1τ], the range of the medium threshold is: (θ1τ, θ2τ], and the range of the large threshold is: (θ2τ, ∞), where τ is the total time spent by the first picking robot to complete the apple picking task for a single hedge, θ1 is the equilibrium multiple coefficient of the small threshold, and θ2 is the equilibrium multiple coefficient of the medium threshold; Determine the implementation principle of the dynamic picking path planning of the picking robots according to the adaptive function for dynamic picking of the two-sided picking robots; Guide the apple picking methods of the two-sided picking robots according to the implementation principle of the dynamic picking path planning of the picking robots.

7. The intelligent path planning method of an apple picking robot based on deep learning according to claim 6, characterized in that, The specific steps for establishing an intelligent path planning model based on the deep learning algorithm model, combining the predicted value of the dynamic delay coefficient and the adaptive function for dynamic picking, and obtaining the optimal path planning of the apple picking robot include: Extract the feature items of the intelligent path planning of the apple picking robot according to the predicted value of the dynamic delay coefficient, the adaptive function, and the implementation principle of the path planning; Based on big data or test experiments, obtain the training sample set and the target sample set of the intelligent path planning of the apple picking robot; Based on the deep learning algorithm model, establish an intelligent path planning model, and obtain the optimal path planning of the apple picking robot by real-time obtaining the predicted value of the dynamic delay coefficient and the adaptive function for dynamic picking of the two-sided picking robots, as well as the judgment situation of the implementation principle of the dynamic picking path planning of the picking robots.

8. The intelligent path planning method of an apple picking robot based on deep learning according to claim 7, characterized in that, The established human-machine interaction platform is used to store, display, and give early warnings about the monitoring data of the apple picking robot during the picking process, and the initial path planning specifically includes: Establish a human-machine interaction platform. Based on the Internet of Things technology, receive and store the monitoring data of the two-end picking robot during the picking process; Based on the human-machine interaction platform, build the operating environment of the intelligent path planning model, and update and optimize the practicability of the model by optimizing parameters; Based on the human-machine interaction platform, display and give early warnings about the position information and abnormal situations of the two-end picking robot during the apple picking process; Based on the human-machine interaction platform, complete the initial path planning and abnormal manual remote control of the two-end picking robot through the human-machine interaction interface.

9. An intelligent path planning system for an apple picking robot based on deep learning, characterized in that, For implementing the intelligent path planning method of the apple picking robot based on deep learning as described in any one of claims 1-8, including: A picking distribution acquisition module, which is used to obtain the channel spacing between adjacent two hedges, the length of each hedge, and the number of hedges in the picking area according to the apple planting distribution situation; A picking method module, which is used to adopt a complementary apple picking method to pick the apples on both sides of the apple tree, and improve the apple picking efficiency and reduce the influence of light through double-machine cooperation; A path planning module, which is used to obtain the predicted value of the dynamic delay coefficient of apple picking in a single-frame area of the two-side picking robot according to the picking efficiency of the apple picking robot; obtain the adaptive function of the two-side picking robot's dynamic picking according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process; determine the implementation principle of the dynamic picking path planning of the picking robot according to the adaptive function of the two-side picking robot's dynamic picking; based on the deep learning algorithm model, combine the predicted value of the dynamic delay coefficient and the dynamic picking adaptive function to establish an intelligent path planning model, and obtain the optimal path planning of the apple picking robot; An interaction platform module, which is used to establish a human-machine interaction platform for storing, displaying, and giving early warnings about the monitoring data of the apple picking robot during the picking process, and the initial path planning.

10. The intelligent path planning system of an apple picking robot based on deep learning according to claim 8, characterized in that, The path planning module includes: A dynamic delay unit, which is used to obtain the predicted value of the dynamic delay coefficient of apple picking in a single-frame area of the two-side picking robot according to the picking efficiency of the apple picking robot; An adaptive function unit, which is used to obtain the adaptive function of the two-side picking robot's dynamic picking according to the predicted value of the dynamic delay coefficient and the sudden obstacle situation during the picking process; An implementation principle unit, which is used to determine the implementation principle of the dynamic picking path planning of the picking robot according to the adaptive function of the two-side picking robot's dynamic picking; A path planning unit, which is used to establish an intelligent path planning model based on the deep learning algorithm model, combine the predicted value of the dynamic delay coefficient and the dynamic picking adaptive function, and obtain the optimal path planning of the apple picking robot.

Images