Motion planning method and system for picking robot arm

Abstract

The present invention provides a picking robot arm motion planning method and system, and relates to the technical field of agricultural picking. The present invention collects information of mature fruits and vegetables in an area to be picked to generate first picking information M1, uses a drone to perform aerial collection of visual image data in the area to be picked to generate second picking information M2; performs comparison and analysis of mature fruit and vegetable information to generate information data of mature fruits and vegetables blocked by branches and leaves, collects surrounding environment information of mature fruits and vegetables blocked by branches and leaves and generates third picking information M3; integrates the collected data in the first picking information M1, the second picking information M2 and the third picking information M3, and generates a picking evaluation model P after analyzing and processing, generates an initial planning path H1 based on a value range set of the picking evaluation model P, generates a branch and leaf occlusion evaluation index ZDZs, judges the picking motion planning of a fruit and vegetable picking robot arm by a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs, and generates a planning path H2 and a planning path H3 in sequence.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural picking, and in particular to a motion planning method and system for a picking robot arm. Background Art

[0002] The motion planning of the fruit and vegetable picking robot involves methods such as inverse kinematics, trajectory generation, collision detection and avoidance, path planning, and force / torque control. All of these methods are based on the use of visual cameras to obtain the position information of mature fruits and vegetables before proceeding to subsequent picking steps. Inverse kinematics is used to solve joint angles, generate smooth trajectories, detect and avoid collisions, plan effective paths, and achieve flexible control, ensuring that the robot moves efficiently, safely, and accurately during picking tasks and adapts to the shapes and hardness of different fruit and vegetable plants.

[0003] For example, Chinese patent application publication number CN115139315A discloses a grasping motion planning method for a harvesting robot arm. By partitioning the robot arm's workspace and defining a pre-picking plane, the entire harvesting planning process is divided into two parts: offline planning and online planning. The offline planning part is trained using a reinforcement learning algorithm, the PPO algorithm. First, the fruit's position is initially identified based on a wide-angle perspective. The fruit's position is projected onto the pre-picking plane to determine the pre-picking points. Reinforcement learning is then used to output the trajectory reaching each pre-picking point. The offline and online planning parts are then modeled as a traveling salesman problem, planning the optimal picking sequence. Finally, the robot arm arrives at the pre-picking points in sequence according to the planned picking sequence, using a close-up perspective to identify the fruit stems and complete the harvest. Using offline planning and a planned picking sequence, the robot arm significantly improves the success rate and efficiency of the harvesting robot.

[0004] The shortcomings are as follows: the Chinese patent application with publication number CN115139315A divides the working space of the robotic arm to realize offline and online planning of the picking path. However, the surrounding environment of mature fruits and vegetables is often blocked by branches and leaves. It is difficult to identify the obscured mature fruits and vegetables by relying solely on the visual recognition camera on the robotic arm. In addition, in the picking path planning for mature fruits and vegetables obscured by branches and leaves, there are cases where the robotic arm is scratched or stuck by branches and leaves. How to collect visual images of picked fruits and vegetables, analyze the position information of mature fruits and vegetables obscured by branches and leaves, and construct a picking evaluation model and a branch and leaf occlusion evaluation index respectively. Based on the value range set of the picking evaluation model, the pre-picking path between the robotic arm and the target fruit is planned, the information of the obscured mature fruits and vegetables is determined, and the initial planned path H1 is generated; then, based on the comparison of the branch and leaf occlusion evaluation index with the set threshold, it is judged whether the fruit and vegetable picking robotic arm can directly ignore the branch and leaf occlusion when moving to plan the fruit and vegetable picking paths H2 and H3;

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the Invention

[0006] The purpose of the present invention is to provide a picking robot arm motion planning method and system to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] The motion planning method of the picking robot arm includes the following steps:

[0009] S1. Obtain historical picking data and a botanical database for the area to be picked, and use them to identify key information about the growth cycle and maturity characteristics of different fruits and vegetables; collect information about mature fruits and vegetables along the overall movement path of the picking robot arm in the area to be picked, and generate first picking information M1;

[0010] S2. Using a drone equipped with a visual recognition camera to collect visual image data of the area to be picked, and adjusting the direction of the wind generated by the rotation of the drone's rotors to reveal fruits and vegetables obscured by branches and leaves, and generating second picking information M2 for picking ripe fruits and vegetables over a large area; after obtaining the first picking information M1 and the second picking information M2, a comparison analysis of the ripe fruits and vegetables is performed to generate information data of ripe fruits and vegetables obscured by branches and leaves;

[0011] S3. Collect environmental information about the mature fruits and vegetables that are obscured by branches and leaves and generate third picking information M3. Fusion and analysis of the collected data in the first picking information M1, the second picking information M2, and the third picking information M3 generate a picking evaluation model P, determine a value range set for the picking evaluation model P, and plan a pre-picking path between the robotic arm and the target fruit based on the value range set of the picking evaluation model P to generate an initial planned path H1.

[0012] S4. After obtaining the first picking information M1, the second picking information M2, and the third picking information M3, a branch and leaf occlusion evaluation index ZDZs is generated. A comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore branch and leaf occlusion during picking motion planning, and a planned path H2 and a planned path H3 are generated in sequence.

[0013] S5. Continuously obtain environmental change data of the picking robot arm during the picking process, set an emergency response mechanism for the picking robot arm to escape from a stuck state, and determine whether new obstructions appear or the existing obstruction status changes to determine whether to trigger the emergency response mechanism.

[0014] A picking robot arm motion planning system, the system is used to execute the picking robot arm motion planning method, comprising:

[0015] The first picking information establishment module: obtains historical picking data and a botanical database of the area to be picked, and uses them to identify key information about the growth cycle and maturity characteristics of different fruits and vegetables; collects information about mature fruits and vegetables along the overall movement path of the picking robot arm in the area to be picked, and generates first picking information M1;

[0016] Second picking information establishment module: A drone equipped with a visual recognition camera collects visual image data of the picking area from the air, adjusts the wind direction generated by the drone's rotor rotation, reveals fruits and vegetables obscured by branches and leaves, and generates second picking information M2 for picking ripe fruits and vegetables over a large area. After obtaining the first picking information M1 and the second picking information M2, a comparison analysis of the ripe fruits and vegetables is performed to generate information data on ripe fruits and vegetables obscured by branches and leaves.

[0017] The picking evaluation model construction module collects environmental information about ripe fruits and vegetables obscured by branches and leaves and generates third picking information M3. The module integrates the collected data from the first picking information M1, the second picking information M2, and the third picking information M3, analyzes and processes the data, generates a picking evaluation model P, and determines the value range set of the picking evaluation model P. Based on the value range set of the picking evaluation model P, the module plans the pre-picking path between the robotic arm and the target fruit to generate the initial planned path H1.

[0018] Branch and leaf occlusion evaluation index establishment module: After obtaining the first picking information M1, the second picking information M2, and the third picking information M3, a branch and leaf occlusion evaluation index ZDZs is generated, and a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore branch and leaf occlusion during picking motion planning, and the planned paths H2 and H3 are generated in sequence;

[0019] Emergency response mechanism feedback module: Continuously obtains environmental change data of the picking robot arm during the picking process, sets an emergency response mechanism for the picking robot arm to escape from the stuck state, and judges whether there is new occlusion or the existing occlusion status changes to determine whether to trigger the emergency response mechanism.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] By collecting information on mature fruits and vegetables in the area to be picked, first picking information M1 is generated, and a drone equipped with a visual recognition camera is used to perform aerial collection of visual image data of the area to be picked to generate second picking information M2. After obtaining the first picking information M1 and the second picking information M2, the mature fruit and vegetable information is compared and analyzed to generate information data of mature fruits and vegetables blocked by branches and leaves, and the surrounding environment information of mature fruits and vegetables blocked by branches and leaves is collected to generate third picking information M3. The collected data in the first picking information M1, the second picking information M2, and the third picking information M3 are integrated and analyzed to generate a picking evaluation model P. Based on the value range set of the picking evaluation model P, a pre-picking path between the robot arm and the target fruit is planned to generate an initial planned path H1 and a branch and leaf occlusion evaluation index ZDZs. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to judge whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion during picking motion planning, and the planned paths H2 and H3 are generated in sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the overall method flow of the present invention;

[0023] Figure 2 is a system block diagram of the present invention;

[0024] Figure 3 This is a schematic diagram of the planning of H1, H2 and H3 of the present invention. DETAILED DESCRIPTION

[0025] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to specific embodiments.

[0026] It should be noted that, unless otherwise defined, the technical or scientific terms used in the present invention should have the usual meanings understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0027] See also Figure 1 、 Figure 2 and Figure 3 , the present invention provides a technical solution:

[0028] Example 1:

[0029] A picking robot arm motion planning method includes the following steps:

[0030] S1. Obtain historical picking data and a botanical database for the area to be picked, and use them to identify key information about the growth cycle and maturity characteristics of different fruits and vegetables; collect information about mature fruits and vegetables along the overall movement path of the picking robot arm in the area to be picked, and generate first picking information M1;

[0031] S2. Using a drone equipped with a visual recognition camera to collect visual image data of the area to be picked, and adjusting the direction of the wind generated by the rotation of the drone's rotors to reveal fruits and vegetables obscured by branches and leaves, and generating second picking information M2 for picking ripe fruits and vegetables over a large area; after obtaining the first picking information M1 and the second picking information M2, a comparison analysis of the ripe fruits and vegetables is performed to generate information data of ripe fruits and vegetables obscured by branches and leaves;

[0032] S3. Collect environmental information about the mature fruits and vegetables that are obscured by branches and leaves and generate third picking information M3. Fusion and analysis of the collected data in the first picking information M1, the second picking information M2, and the third picking information M3 generate a picking evaluation model P, determine a value range set for the picking evaluation model P, and plan a pre-picking path between the robotic arm and the target fruit based on the value range set of the picking evaluation model P to generate an initial planned path H1.

[0033] S4. After obtaining the first picking information M1, the second picking information M2, and the third picking information M3, a branch and leaf occlusion evaluation index ZDZs is generated. A comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore branch and leaf occlusion during picking motion planning, and a planned path H2 and a planned path H3 are generated in sequence.

[0034] S5. Continuously obtain environmental change data of the picking robot arm during the picking process, set an emergency response mechanism for the picking robot arm to escape from a stuck state, and determine whether new obstructions appear or the existing obstruction status changes to determine whether to trigger the emergency response mechanism.

[0035] Example 2:

[0036] Further explanation based on the first embodiment, the acquisition of historical picking data and botanical database of the area to be picked, and the use thereof to identify key information of growth cycles and maturity characteristics of different fruits and vegetables, specifically includes the following:

[0037] The historical picking data includes the date, time, yield, and variety of fruit and vegetable picking in the area to be picked in the past;

[0038] Usage: By analyzing this data, we can predict the optimal picking period for different fruits and vegetables, as well as information such as expected yield.

[0039] The botanical database contains the growth cycles and maturity characteristics of different fruits and vegetables, including color, size, shape, and optimal picking time;

[0040] Purpose: Provides decision support on which fruits and vegetables to plant and when, and helps identify ripe fruits and vegetables.

[0041] The picking robot collects information on mature fruits and vegetables along the entire moving path of the picking area and generates first picking information M1, which specifically includes the following contents:

[0042] The first picking information M1 is data obtained by an external visual sensor, specifically including the following collected data:

[0043] Fruit and vegetable location and maturity data: External visual sensors capture the precise location information Wz1 and maturity information CSd of fruits and vegetables. Maturity information CSd is obtained by collecting color C1, size C2, and shape C3 data of fruits and vegetables in the image and performing function calculations. The location information Wz1 includes GPS coordinates of latitude, longitude, and altitude.

[0044] Fruit and vegetable type and characteristic data: Based on the type characteristics LXTz of different fruits and vegetables, record their specific growth cycle and maturity characteristics;

[0045] The current position and status data of the robot arm: including the robot arm's coordinates Zb, direction Fx, and the current task status ZT data; the robot arm is equipped with a variety of sensors that can monitor its coordinates, direction and current task status in real time, which are obtained through built-in sensors and control systems;

[0046] The formula for the first picking information M1 is:

[0047]

[0048] where ln, sin, exp represent natural logarithm, square root, sine, and exponential functions respectively; α and β are adjustment parameters;

[0049] Wz1=(GPS lat ,GPS lon ,GPS alt )

[0050] Combine the elements of Wz1 into a scalar value. Specifically, use the average of longitude, latitude, and altitude to represent the scalar value of Wz1. The calibration formula is as follows:

[0051]

[0052] Among them, Wz1 represents the precise location information of fruits and vegetables, including longitude GPS lat , latitude GPS lon and altitude GPS alt ;

[0053] CSd=k1·C1+k2·C2+k3·C3

[0054] Among them, CSd represents maturity information, which is a function based on the color C1, size C2 and shape C3 of fruits and vegetables. k1, k2 and k3 are weight parameters set according to actual conditions, which are used to adjust the contribution of color, size and shape to maturity assessment.

[0055] Set the range of M1 to [0.1,1]:

[0056] When M1 is close to 0.1, it indicates that the picking efficiency or accuracy is low; this is due to unclear fruit and vegetable position Wz1, inaccurate maturity assessment CSd, or unsatisfactory coordinates / direction / task status of the robot arm (Zb, Fx, ZT);

[0057] When M1 is close to 1, it indicates that the picking efficiency or accuracy is high, which means that all parameters are in an ideal state.

[0058] Example 3:

[0059] Further explanation is given on the basis of the first embodiment. The drone equipped with a visual recognition camera collects visual image data of the area to be picked from the air, and adjusts the direction of the wind generated by the rotation of the drone's rotor to reveal the fruits and vegetables blocked by branches and leaves, and generates second picking information M2 for picking mature fruits and vegetables in a large range. After obtaining the first picking information M1 and the second picking information M2, a comparison analysis of the mature fruits and vegetables is performed to generate information data of mature fruits and vegetables blocked by branches and leaves, which specifically includes the following contents:

[0060] Monitor the impact of the wind generated by the drone's rotating rotors on surrounding vegetation; adjust the drone's flight altitude and angle, or adjust the rotor speed, to generate appropriate wind force to expose fruits and vegetables hidden by branches and leaves;

[0061] While adjusting the wind direction, the drone's onboard visual recognition camera continues to capture images, identifying fruits and vegetables exposed by the wind. The collected image data is then sent to a processing center for image analysis to identify ripe fruits and vegetables.

[0062] Adjust the flight altitude and angle of the drone, or adjust the rotor speed. The specific steps are as follows;

[0063] The flight height adjustment quantization step sets the adjustment range of the drone's flight height H to H min To H max , this range should be based on the safe flight altitude of the UAV and the average height of the target crops. The adjustment of the flight altitude can be expressed by the following formula:

[0064]

[0065] Where H is the flight altitude of the UAV;

[0066] S is the occlusion degree score obtained from the visual system, and the score range is set from 1 to 10. The higher the score, the more severe the occlusion.

[0067] Among them, 1 means 0%-10% occlusion, 2 means 11%-20% occlusion, 3 means 21%-30% occlusion, 4 means 31%-40% occlusion, 5 means 41%-50% occlusion, 6 means 51%-60% occlusion, 7 means 61%-70% occlusion, 8 means 71%-80% occlusion, 9 means 81%-90% occlusion, and 10 means 91%-100% occlusion;

[0068] Flight angle adjustment quantization;

[0069] Set the flight angle θ of the drone to θ min to θ max , the adjustment of the flight angle can quantify the direction of the branch and leaf occlusion area as needed:

[0070]

[0071] Where θ is the flight angle of the UAV relative to the horizontal plane;

[0072] D is the target direction recognized by the visual system, such as north, south, east, west or angles between them, converted into a value from 0 to D max The value of

[0073] D max It is the maximum value of the direction, 360°;

[0074] Quantification of rotor speed adjustment;

[0075] Set the rotor speed R adjustment range to R min to R max ; The rotor speed adjustment can be quantified based on wind demand:

[0076]

[0077] Where: R is the rotor speed;

[0078] F is the wind demand score based on real-time video analysis, ranging from 0 to F max The higher the score, the stronger the wind force required to remove the obstruction. The wind force requirement score is proportional to the obstruction degree score. That is, the higher the obstruction degree, the higher the required wind force level. When the obstruction score is set to 1, the initial wind force level required is 5 Newtons. Each increase in the wind force level increases by 10%.

[0079] For soft leaves, such as lettuce and spinach, when the shading score is 1, the required initial wind force level is 5 Newtons, and the wind force increases by 10% for each additional level.

[0080] For leaves of medium thickness and hardness, such as carrot leaves and celery leaves, when the shading score is 1, the required initial wind force level is 10 Newtons, and the wind force increases by 15% for each additional level.

[0081] For thicker and harder leaves, such as cabbage leaves and radish leaves, when the shading score is 1, the required initial wind force level is 20 Newtons, and the wind force increases by 12% for each additional level.

[0082] F max is the maximum value of the wind demand score, and the adaptive improvement is made according to the leaf type of the picked fruits and vegetables. max The wind force represents the ability to move the corresponding leaf surface until its lateral displacement reaches 80% of the original leaf width.

[0083] The second picking information M2 specifically includes the following collected data:

[0084] Wide-area fruit and vegetable distribution data: The drone provides a wide-area distribution map of the area to be harvested, including the group distribution QTFb of fruits and vegetables and the row spacing HJj;

[0085] The drone uses the wind generated by its rotor blades to reveal information about fruits and vegetables that are obscured by branches and leaves. Specific data includes:

[0086] Occlusion location information: the latitude, longitude, and altitude of the obscured fruits and vegetables in the farmland;

[0087] Visible part features KJBf: Features of the visible area after the fruits and vegetables are occluded. Visible area features include D1 color, D2 size, and D3 shape;

[0088] The ZDCs indicate the degree to which fruits and vegetables are obscured, such as complete or partial obscuration. The ZDCs are divided into 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 levels, with larger values ​​representing larger obscured areas.

[0089] 1 means 0%-10% obscured, 2 means 11%-20% obscured, 3 means 21%-30% obscured, 4 means 31%-40% obscured, 5 means 41%-50% obscured, 6 means 51%-60% obscured, 7 means 61%-70% obscured, 8 means 71%-80% obscured, 9 means 81%-90% obscured, and 10 means 91%-100% obscured;

[0090] The above grade settings are critical for determining whether branches and leaves need to be moved for picking;

[0091] Fruit and vegetable group information GSQt: data on the obscured conditions of multiple fruits and vegetables in the area;

[0092] Branch and leaf motion data YDSj: The movement of branches and leaves caused by the wind force of the UAV rotor, including the movement amplitude YDFd and frequency f of the branches and leaves;

[0093] Combining external visual sensors and drone data, using data fusion algorithms such as Kalman filters to optimize the accuracy of fruit and vegetable location and maturity information, and building a multi-level decision tree model to analyze and determine picking priorities;

[0094] The formula for the second picking information M2 is:

[0095]

[0096] Among them, e x ,tanh(x),Γ(x),x y ,ln(x) represents exponential, hyperbolic tangent, gamma function, power function and natural logarithm respectively;

[0097] QTFb represents the wide-area distribution of fruit and vegetable groups;

[0098] HJj represents the row spacing in the farmland;

[0099] The formula for the position information Wz2 of the obscured fruits and vegetables is:

[0100] Wz2=(GPS lat2 ,GPS lon2 ,GPS alt2 )

[0101] The characteristic KJBf formula of the visible part of fruits and vegetables is:

[0102] KJBf=k4·D1+k5·D2+k6·D3

[0103] The parameters are explained as follows: color D1, size D2, shape D3, k4, k5 and k6 are weight parameters set according to actual conditions, which are used to adjust the contribution of color, size and shape to the visible features of fruits and vegetables;

[0104] ZDCs represents the degree to which fruits and vegetables are shaded, for example, ZDCs = 5 means 50% shaded;

[0105] The YDFd calculation formula is:

[0106]

[0107] It represents the movement of branches and leaves caused by the wind force of the UAV rotor, including the movement amplitude YDFd and frequency f;

[0108] Set the range of M2 to [0,10]:

[0109] When M2 is close to 0, it indicates that the recognition and positioning capabilities in complex environments are poor, because fruits and vegetables are severely blocked by ZDCs or the movement amplitude of branches and leaves is large YDSj;

[0110] When M2 is close to 10, it indicates strong ability, which is conducive to identification and positioning through the distribution of fruit and vegetable groups QTFb and row spacing HJj;

[0111] Example 4:

[0112] Further illustrating on the basis of the first embodiment, the method of collecting the surrounding environment information of the ripe fruits and vegetables blocked by branches and leaves and generating the third picking information M3 specifically includes the following contents:

[0113] The third picking information M3 includes integrated meteorological data QXSj, lighting condition data GZTj, vegetation density ZBMd and historical picking data LSCz:

[0114] The integrated meteorological data QXSj specifically collects the temperature Wd, humidity Sd, wind speed Fs, and light condition GZTj data around mature fruits and vegetables that are blocked by branches and leaves. The integrated meteorological data QXSj is obtained through a weather station or a mobile meteorological sensor.

[0115] The light condition data GZTj include sunshine duration t1 and sun position tw;

[0116] The formula for the third picking information M3 is:

[0117]

[0118] Among them, Φ(x), log b(a) represent the standard normal distribution function, cube root, and logarithm of a with base b, respectively;

[0119] Set the value range of M3 to [0.5,5];

[0120] When M3 is close to 0.5, it means that the current climate and environmental conditions are not conducive to picking, such as unfavorable climate data, insufficient light conditions or too high vegetation density;

[0121] When M3 is close to 5, it means that the climate and environmental conditions are very favorable for picking, such as ideal meteorological conditions, sufficient sunlight and moderate vegetation density, as well as a good match between historical data and current conditions;

[0122] Interactive content supplement:

[0123] Under different climate and environmental conditions, such as different QXSj, GZTj, ZBMd, and LSCz, the value of M3 will change, thus affecting the formulation of the picking strategy. For example, under poor climate conditions, QXSj is not ideal, and it is necessary to adjust the picking time or use machinery that is more adaptable to inclement weather. On the other hand, under conditions with high light conditions GZTj and moderate vegetation density ZBMd, picking efficiency can be greatly improved.

[0124] Changes in M3 values ​​can also provide guidance on how to better utilize historical harvest data LSCz; if historical data shows that harvesting is more effective under specific climate conditions, then those successful strategies can be repeated under similar conditions;

[0125] Real-time monitoring and analysis of M3 can help decision-makers respond more quickly to environmental changes and optimize harvesting plans, such as picking earlier when adverse weather conditions are expected or expanding the harvest area under ideal conditions;

[0126] Furthermore, by analyzing historical trends and patterns in M3, future harvesting conditions can be predicted, allowing for advance planning and preparation to address challenges as they arise;

[0127] The steps of fusing the collected data in the first picking information M1, the second picking information M2 and the third picking information M3 and analyzing and processing the data to generate the picking evaluation model P, and determining the value range set of the picking evaluation model P, specifically include the following:

[0128] Set the picking evaluation model P as:

[0129]

[0130] in:

[0131] Wz1: represents the precise location information of fruits and vegetables;

[0132] CSd: represents maturity information, which is calculated from the color C1, size C2, and shape C3 of fruits and vegetables;

[0133] The specific calculation steps of CSd are as follows: make judgment based on the historical harvest data of the area to be harvested and the botanical database;

[0134] Color C1: Represents the color characteristics of fruits and vegetables. Different types of fruits and vegetables will show different colors at different stages of maturity. For example, a tomato turns from green to red when it is ripe.

[0135] Size C2: represents the physical size of fruits and vegetables. Fruits and vegetables will gradually grow larger during the growth process, and reaching a certain size usually means they are mature.

[0136] Shape C3: represents the shape characteristics of fruits and vegetables. Some fruits and vegetables will change in shape when they mature, such as bananas, which will change from straight to curved;

[0137] Construction of function f:

[0138] Data collection: First, we need to collect a large amount of data on the color, size, and shape of different types of fruits and vegetables at different levels of maturity;

[0139] Feature quantification: Converting qualitative features such as color, size, and shape into quantifiable values. For example, color can be represented by RGB values, size can be represented by volume or length, and shape can be quantified by proportion or specific shape metrics.

[0140] Algorithm design: Use machine learning or other statistical methods to train a model based on the collected data. This model will learn the relationship between color, size, and shape and maturity, and be able to predict the maturity corresponding to a given color, size, and shape.

[0141] Calculation of maturity

[0142] Input feature values: Extract the color C1, size C2, and shape C3 values ​​of fruits and vegetables from the collected images.

[0143] Apply function f: Take these values ​​as input and apply them to function f to get the predicted value of maturity CSd.

[0144] Maturity judgment: Based on the calculated CSd value, determine whether the fruits and vegetables have reached the maturity for picking.

[0145] LXTz: represents the type and characteristics of fruits and vegetables;

[0146] Zb, Fx, ZT: represent the coordinates, direction and current task status of the robot arm respectively;

[0147] M1, M2, M3: represent the first, second and third picking information respectively;

[0148] f(i) is an advanced integral function, representing a comprehensive analysis of environmental factors;

[0149] YDFd and GSQt represent the movement amplitude of branches and leaves and the information of fruit and vegetable groups, respectively;

[0150] QTFb, HJj, Wz2, KJBf, and ZDCs represent the distribution of fruit and vegetable groups, row spacing, occlusion position information, characteristics of the visible part, and occlusion degree, respectively;

[0151] The advanced integral function f(i) is defined as:

[0152]

[0153] This function is an integral of a Gaussian function, which is used to simulate the impact of environmental factors on the picking path;

[0154] The value range of the picking evaluation model P is (0, +∞), and different values ​​represent different picking efficiencies and path optimization levels. A lower picking evaluation model P value indicates higher picking efficiency and more optimized path planning, while a higher picking evaluation model P value means that the picking strategy or path planning needs to be improved.

[0155] The pre-picking path between the robot arm and the target fruit is planned based on the value range set of the picking evaluation model P to generate an initial planning path H1, which specifically includes the following contents:

[0156] Set the interval values ​​A and B of the picking evaluation model P, and A is less than B, and divide the value interval of the picking evaluation model P into a high efficiency interval (0, A), a medium efficiency interval (A, B) and a low efficiency interval (B, +∞);

[0157] The high-efficiency interval (0, A) represents very high picking efficiency and optimized paths. The path planning principle is to quickly respond to significant changes in environmental factors f(i) and generate the initial planned path H1.

[0158] The medium efficiency interval (A, B) indicates that the picking efficiency is at a medium level. The path planning principle is to set up a feedback mechanism to analyze whether the efficiency decreases due to the fruit and vegetable type characteristics LXTz or the branch and leaf movement amplitude YDFd, and make corresponding adjustments to generate the initial planned path H1;

[0159] The low efficiency interval (B, +∞) indicates that the picking efficiency is at a low level. The path planning principle is to set up a deep analysis process, focusing on analyzing the impact of the occlusion degree ZDCs and the occlusion position information Wz2. If the measurement results are in this interval for several consecutive times, the picking strategy or path planning is completely redesigned to generate the initial planned path H1. In this embodiment, the number of times is set to 3.

[0160] Specifically, the high-efficiency interval (0, A): Since this interval represents very high picking efficiency and optimized path, A is a relatively small value. In practical applications, this means that the robot arm moves very accurately and efficiently without unnecessary movements. Setting A to a small positive number, such as 10, means that when the value of the picking evaluation model P is less than 10, we consider the picking path to be highly optimized.

[0161] Adaptability to environmental factors: In the high efficiency range, if a significant change in the environmental factor f(i) value is detected, such as a sudden change in light or temperature fluctuation, the system can quickly adjust to keep the picking evaluation model P value within this range;

[0162] Specifically: In the high efficiency range, the system must be able to respond quickly to significant changes in environmental factors f(i); taking light changes and temperature fluctuations as examples:

[0163] Adaptability to light changes: Set a light intensity threshold, such as 500-700 lbf / s. If the sensor detects that the light intensity drops below or exceeds this range, the system will adjust the robot's speed and path to adapt to the light change. For example, if the light intensity suddenly increases to 800 lbf / s, the robot will need to slow down to prevent overexposure from affecting its visual recognition system.

[0164] Medium efficiency interval (A, B): This interval indicates that the picking efficiency is at a medium level. B is set to be greater than A, and it can distinguish between medium efficiency and low efficiency. In this interval, the robot arm needs to perform some path optimization and parameter adjustment, but these adjustments should not be too complex. Taking these factors into consideration, B is set to a medium value, such as 50. This means that when the value of the picking evaluation model P is between 10 and 50, the robot arm's picking efficiency is at a medium level.

[0165] Feedback adjustment mechanism: In the medium efficiency range, if the P value of the picking evaluation model continues to fluctuate near the upper limit of the range, the system should activate the feedback mechanism to analyze whether the efficiency decline is due to the fruit and vegetable type characteristics LXTz or the branch and leaf movement amplitude YDFd, and make corresponding adjustments;

[0166] Specifically, in the medium efficiency range, the continuous fluctuation of the P value of the picking evaluation model indicates that adjustments are needed; the following is an example of the implementation of this mechanism:

[0167] LXTz Adjustment: For example, a fruit or vegetable becomes softer as it ripens, which affects the robot's gripping efficiency. If the P value of the picking evaluation model consistently fluctuates between 40 and 50, the system will identify the need to adjust the gripping pressure. For example, if the robot is handling soft fruits and vegetables, the pressure needs to be reduced to avoid damaging the fruit.

[0168] YDFd adjustment: If the movement of branches and leaves increases due to increased wind speed, the robot's path planning needs to be adjusted. For example, if the wind speed increases from 5 km / h to 15 km / h, causing the P value of the picking evaluation model to rise to 45, the system needs to increase the redundancy of the path planning to adapt to this change.

[0169] Low efficiency range: When the value of the picking evaluation model P is greater than 50, picking efficiency is low. This means that the robot's movement path needs to be significantly adjusted, or environmental factors have a significant impact on picking efficiency. In this range, picking path planning faces significant challenges and requires in-depth analysis and redesign.

[0170] In-depth analysis and redesign: In low-efficiency intervals, the system should trigger an in-depth analysis process, focusing on analyzing the impact of occlusion levels ZDCs and occlusion position information Wz2. If several consecutive measurement results remain in this interval, the picking strategy or path planning should be fully redesigned.

[0171] Specifically, in the low-efficiency range, the system needs to be deeply analyzed for redesign; the following is a specific implementation plan:

[0172] Analyze the impact of ZDCs: If the P-value of the picking evaluation model is higher than 50 for several consecutive pickings, it indicates a serious occlusion problem. For example, if the P-value of the picking evaluation model is continuously higher than 60 when picking fruits and vegetables with dense leaves, it indicates that the occlusion level is too high, resulting in the robot's vision system being unable to effectively identify the fruits and vegetables. Solutions include improving the vision system or adjusting the robot's motion path to better handle high-occlusion environments.

[0173] Analyze the impact of Wz2: If the occlusion position information Wz2 is detected to have a significant impact on efficiency, for example, if the P value of the picking evaluation model exceeds 60 for several consecutive measurements and the occlusion position changes frequently, it indicates that the path planning needs to be redesigned. For example, the robot arm needs to adopt more flexible path planning to adapt to frequently changing occlusion situations, or improve its ability to recognize occlusions to more accurately adjust the path.

[0174] Embodiment 5:

[0175] Further explanation is given based on the fourth embodiment.

[0176] In order to verify and demonstrate the effectiveness of the invention, this embodiment is designed to test three efficiency ranges: a high efficiency range (0, 10), a medium efficiency range (10, 50), and a low efficiency range (50, +∞); these tests will be conducted for different environmental factors and types of fruits and vegetables;

[0177] Experimental preparation

[0178] Equipment and Materials: Use advanced harvesting robots with multiple sensors and adjustable robotic arms; prepare a variety of fruit and vegetable types, including soft and hard fruits and vegetables, and different densities and positions of obstructions such as leaves and branches;

[0179] Environment settings: Create multiple controlled environments to simulate different lighting and wind speed conditions; for example, set up an indoor environment where the lighting can be adjusted from 200 loads to 1000 loads; and the wind speed can be varied from 0 to 20 km / h;

[0180] Data collection system: Prepare an advanced data recording system to record in real time the values ​​of the picking evaluation model P, environmental factors f(i), such as light intensity, temperature, and wind speed, as well as the type and characteristics of fruits and vegetables LXTz and the amplitude of branch and leaf movement YDFd;

[0181] Implementation process:

[0182] High-efficiency interval test: Set the light intensity and temperature within the normal range, such as 500-700 loads; start the picking operation and record the P value of the picking evaluation model; during the test, suddenly change the light intensity to 800 loads, observe and record the robot arm's response and the change in the P value of the picking evaluation model;

[0183] Medium efficiency range test: Adjust light and temperature settings to keep the P value of the picking evaluation model between 10 and 50; introduce fruits and vegetables of varying hardness and simulate changes in wind speed, such as increasing from 5 km / h to 15 km / h; record the adjustments to the robotic arm and the changes in the P value of the picking evaluation model;

[0184] Low-efficiency interval test: Increase the density and complexity of the obstructions so that the P value of the picking evaluation model exceeds 50; record the occlusion degree ZDCs and obtain the following experimental data table:

[0185] The following table shows the specific data of the experiment:

[0186]

[0187]

[0188] Table 1

[0189] Data analysis, by comparing the experimental data in the above table, we can clearly see the effectiveness and beneficial effects of the invention:

[0190] Adaptability to light changes: When the light intensity gradually increased from 500 loads to 800 loads, the initial picking evaluation model P value for ordinary fruits and vegetables was 9, indicating that the robotic arm was in the high efficiency range. When the light intensity suddenly increased, the picking evaluation model P value increased from 9 to 10. Although the increase was slight, it was still in the high efficiency range, indicating that the robotic arm can effectively adapt to light changes and maintain a high picking efficiency. This shows that the light change adaptation mechanism in the invention can ensure that the robotic arm can maintain high picking efficiency when the ambient light changes significantly.

[0191] Adjustment mechanism for wind speed changes: When the wind speed increased from 5 km / h to 15 km / h, the P value of the picking evaluation model increased from 45 to 48 when handling soft fruits and vegetables. Although there was an increase, it was still within the medium efficiency range. This shows that even under the influence of wind speed changes, the system can adapt to environmental changes by adjusting path planning and grasping strategies to maintain a medium level.

[0192] Example 6:

[0193] Further explanation is given on the basis of Example 1. After obtaining the first picking information M1, the second picking information M2 and the third picking information M3, a branch and leaf occlusion evaluation index ZDZs is generated, and a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to judge whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion to pick fruits and vegetables during the picking motion planning, and generate the planned path H2 and the planned path H3 in sequence, which specifically include the following contents:

[0194] Obtain the first picking information M1, the second picking information M2, and the third picking information M3 and generate a branch and leaf occlusion evaluation index ZDZs, and set a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs. The set threshold U of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion to pick fruits and vegetables during movement;

[0195] The calculation formula of the branch and leaf shading evaluation index ZDZs is as follows:

[0196]

[0197] Among them, φ(M1,M2,M3,t) is an integral function based on the picking information, which represents the temporal dynamic change of occlusion during the picking process, and its form is:

[0198] φ(M1,M2,M3,t)=λ1·M1(t)+λ2·sin(M2(t))+λ3·cos(M3(t))

[0199] ω i is the normalized weight, F i (H2, H3) is a complex information filtering function based on path planning, which represents the occlusion situation under different path planning. This function can be specifically expressed as:

[0200]

[0201] Among them, M1, M2, and M3 represent the first, second, and third picking information respectively; t represents the time variable; T is the total duration of the picking process; A is the set threshold; λ1, λ2, and λ3 are adjustment coefficients; H2 and H3 represent two different path plans; N is the number of path plans; ò is a small amount to prevent the denominator from being zero;

[0202] The value range of ZDZs is set to (0, U). The closer ZDZs is to U, the less affected the movement of the fruit and vegetable picking robot arm is in an occluded environment. The optimized planning path H2 is generated. The path planning ignores the occlusion of branches and leaves and takes the shortest path trajectory for mature fruits and vegetables, that is, the shortest path is selected to directly pass through the branch and leaf area.

[0203] If the branch and leaf occlusion evaluation index ZDZs is closer to 0, it is judged that the movement of the fruit and vegetable picking robot arm in the occlusion environment is greatly affected, and it is more inclined to choose a path that avoids the occlusion. The optimized planning path H3 is generated. The path planning focuses on minimizing the contact points of branch and leaf occlusion. For example, the picking path for picking fruit has several intersecting branch and leaf occlusions. Based on the picking evaluation model, a path with the least contact with branch and leaf occlusion is designed to reduce the probability of the fruit and vegetable picking robot arm being stuck by branches and leaves when turning.

[0204] Further explanation: the (0, U) interval is divided into ten equal parts, and the partitions from smallest to largest are Q1, Q2...Q10. Only when the ZDZs value is in the range of Q9 and Q10, the path planning will ignore the occlusion of branches and leaves and take the shortest path trajectory of mature fruits and vegetables. That is, the shortest path will be selected to directly pass through the branch and leaf area, and the corresponding planned path H2 will be generated. The planned path H3 will be generated for the rest of the range.

[0205] Character interpretation:

[0206] e is the base of natural logarithms;

[0207] dt is the integral over time, which represents the cumulative effect of occlusion over time;

[0208] φ is the integral function based on the picking information;

[0209] It is a summation function used to weight the information filtering results of different path planning; ω i is the normalized weight.

[0210] Embodiment seven:

[0211] Further explaining Example 6, this example aims to verify the application effect of the constructed branch and leaf occlusion evaluation index ZDZs on the fruit and vegetable picking robot arm in agricultural automated picking. The experiment is divided into two parts: the first part collects the first, second, and third picking information M1, M2, M3 and calculates the ZDZs; the second part uses the ZDZs to guide the robot arm's path planning, implement the picking task, and record the picking efficiency and success rate.

[0212] Before the experiment began, a fruit and vegetable picking robot equipped with external visual sensors and drone monitoring was prepared. The experimental area was selected to have an environment with foliage of varying density and height to simulate real agricultural picking conditions. First, second, and third picking information was obtained through a real-time monitoring system, sensor data, and historical picking records.

[0213] To calculate ZDZs, the occlusion information during the picking process is first integrated according to the formula φ(M1,M2,M3,t). The occlusion evaluation index is then calculated using the ZDZs formula. A threshold A is set based on the ZDZs value to determine the path planning strategy of the picking robot. When the ZDZs value is high, the robot will choose the shortest path through the branch area; when the ZDZs value is low, the robot will try to avoid branch occlusion.

[0214] In the experiment, the robotic arm performed the picking task based on the ZDZs value while monitoring the frequency of unexpected situations (such as being stuck in branches and leaves). Each time an unexpected situation occurred, the robotic arm would initiate an emergency response mechanism, including reverse movement, occlusion analysis, and path replanning. The experiment recorded the robotic arm's picking efficiency and success rate under different ZDZs values.

[0215] The test form is as follows:

[0216]

[0217]

[0218] Table 2

[0219] Tabular data analysis:

[0220] The experimental data includes the picking efficiency, picking success rate and emergency response times under different ZDZs values; by analyzing these data, we can clearly see the beneficial effects of the invention;

[0221] The relationship between ZDZs value and picking efficiency and success rate:

[0222] Improved efficiency and success rate: When the ZDZs value increases, as in Experiments 5, 6, and 7, the picking efficiency (the number of fruits and vegetables picked per unit time) and the picking success rate (the ratio of successfully picked fruits and vegetables to the total number of fruits and vegetables) show significant improvements. This indicates that with high ZDZs values, the robotic arm can more efficiently traverse the branch and leaf area, reducing detours and time spent searching for harvestable fruit.

[0223] Challenges with Low ZDZs: Conversely, when the ZDZs are low, as in Experiments 1, 2, and 9, both picking efficiency and success rate decrease. This is because low ZDZs mean that occlusions from branches and leaves have a greater impact on the robot's motion, resulting in more obstacles to avoid during path planning, increasing the difficulty and time of picking.

[0224] The relationship between the number of emergency responses and the ZDZs value:

[0225] Reduced emergency responses: At higher ZDZs, such as in Tests 5, 6, and 7, the number of emergency responses was significantly reduced, even to zero. This means the robot encountered fewer obstacles during the picking process, resulting in fewer instances of getting stuck in branches and leaves.

[0226] High-risk low ZDZs values: When the ZDZs values ​​are low, such as in experiments 1, 2, and 9, the number of emergency responses increases, reflecting that the robot arm encounters obstacles and gets stuck in branches and leaves more frequently during the picking process. This further verifies the complexity and risk of path planning under low ZDZs values.

[0227] Verification of the invention by experimental data:

[0228] Optimization of path planning: Experimental data shows that the ZDZs value effectively guides the path planning of the fruit and vegetable picking robot. When the ZDZs value is high, the robot can directly traverse the branches and leaves and choose the shortest path, thereby improving picking efficiency and success rate. This reflects the innovation of the invention in optimizing path planning.

[0229] Dynamic responsiveness: The reduction in emergency response times demonstrates the robotic arm’s efficient response capabilities in real-time dynamic environments.

[0230] Embodiment 8:

[0231] Further explanation based on the first embodiment, the method of continuously acquiring environmental change data of the picking robot arm during the picking process and judging whether new occlusion occurs or the existing occlusion status changes to determine whether to trigger the emergency response mechanism specifically includes the following:

[0232] During execution, external visual sensors and drones are continuously used to monitor environmental changes, such as the appearance of new occlusions or changes in the status of existing occlusions;

[0233] If the robotic arm encounters an accident during execution, such as being caught by branches or leaves, the system will trigger an emergency response mechanism, which includes:

[0234] Reverse movement: The robotic arm attempts to move back slightly to escape from the stuck state;

[0235] Occlusion analysis: Re-evaluate occluded areas to determine if additional mechanical maneuvers, such as rotation or lateral movement, are required to avoid the obstacle;

[0236] Path replanning: Dynamically replan the path based on new occlusion data to find a new optimal path.

[0237] The above formulas are all dimensionless and numerical calculations. The formulas are obtained by collecting a large amount of data and performing software simulation to obtain the most recent real situation. The preset parameters in the formulas are set by those skilled in the art according to actual conditions.

[0238] Specific, ongoing environmental monitoring:

[0239] Sensor deployment: The robotic arm is equipped with various sensors, such as visual sensors, proximity sensors, and tactile sensors, for real-time monitoring of the surrounding environment;

[0240] Data collection: Sensors continuously collect environmental data, including the location, size, and movement status of obstacles;

[0241] Occlusion judgment: By analyzing the collected data, the system can determine whether new occlusions have appeared or whether the status of existing occlusions has changed.

[0242] Emergency Response Mechanism:

[0243] Reverse Movement When the robot arm detects that it may be stuck or is approaching a dangerous area, it will slightly back up to escape the potential risk.

[0244] This action prevents the robotic arm from being damaged or damaging surrounding objects;

[0245] The occlusion analysis system re-evaluates the occluded area to determine if additional mechanical action, such as rotation or sideways movement, is required to avoid the obstacle.

[0246] The analysis is based on the latest sensor data and pre-set algorithms;

[0247] Path Replanning Based on the new occlusion data, the system dynamically replans the path of the robot arm to find a new optimal path to the target location; this includes adjusting the movement trajectory or changing the picking order.

[0248] When the robotic arm itself malfunctions, such as joint jamming or sensor failure, the emergency response should also include automatic shutdown and alarm.

[0249] Characteristics of different crops: Adjust occlusion analysis and path planning strategies based on crop characteristics such as size, shape, and vulnerability.

[0250] Embodiment 9:

[0251] A picking robot arm motion planning system, the system is used to execute the picking robot arm motion planning method, comprising:

[0252] The first picking information establishment module: obtains historical picking data and a botanical database of the area to be picked, and uses them to identify key information about the growth cycle and maturity characteristics of different fruits and vegetables; collects information about mature fruits and vegetables along the overall movement path of the picking robot arm in the area to be picked, and generates first picking information M1;

[0253] Second picking information establishment module: A drone equipped with a visual recognition camera collects visual image data of the picking area from the air, adjusts the wind direction generated by the drone's rotor rotation, reveals fruits and vegetables obscured by branches and leaves, and generates second picking information M2 for picking ripe fruits and vegetables over a large area. After obtaining the first picking information M1 and the second picking information M2, a comparison analysis of the ripe fruits and vegetables is performed to generate information data on ripe fruits and vegetables obscured by branches and leaves.

[0254] The picking evaluation model construction module collects environmental information about ripe fruits and vegetables obscured by branches and leaves and generates third picking information M3. The module integrates the collected data from the first picking information M1, the second picking information M2, and the third picking information M3, analyzes and processes the data, generates a picking evaluation model P, and determines the value range set of the picking evaluation model P. Based on the value range set of the picking evaluation model P, the module plans the pre-picking path between the robotic arm and the target fruit to generate the initial planned path H1.

[0255] Branch and leaf occlusion evaluation index establishment module: After obtaining the first picking information M1, the second picking information M2, and the third picking information M3, a branch and leaf occlusion evaluation index ZDZs is generated, and a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore branch and leaf occlusion during picking motion planning, and the planned paths H2 and H3 are generated in sequence;

[0256] Emergency response mechanism feedback module: Continuously obtains environmental change data of the picking robot arm during the picking process, sets an emergency response mechanism for the picking robot arm to escape from the stuck state, and judges whether there is new occlusion or the existing occlusion status changes to determine whether to trigger the emergency response mechanism.

[0257] The specific values ​​of δ, ε, ∈, and σ in the formula are generally determined by those skilled in the art based on actual conditions. The essence of the formula in the present application is a comprehensive analysis performed by weighted summation. Those skilled in the art collect multiple groups of sample data and set corresponding preset proportional coefficients for each group of sample data; substitute the set preset proportional coefficients and the collected sample data into the formula, and any four formulas constitute a system of four linear equations. The calculated coefficients are screened and averaged to obtain the values ​​of δ, ε, ∈, and σ;

[0258] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. Those skilled in the art will appreciate that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by computer software, electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software depends on the specific application and design constraints of the technical solution.

[0259] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, and may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment as needed.

[0260] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A picking robot arm motion planning method, characterized in that: The specific steps include: S1, obtaining historical picking data and a botanical database of the area to be picked, and using them to identify key information on the growth cycle and maturity characteristics of different fruits and vegetables; collecting information on mature fruits and vegetables on the overall moving path of the picking robot arm in the area to be picked, and generating first picking information M1; The historical harvesting data includes the date, time, yield, and variety of fruit and vegetable harvesting in the area to be harvested in the past; The botanical database contains the growth cycles and maturity characteristics of different fruits and vegetables, where the maturity characteristics are specifically color, size, shape, and the best time to pick them; The picking robot arm collects information about ripe fruits and vegetables on the overall moving path of the area to be picked, and generates first picking information M1, which specifically includes the following contents: The first picking information M1 is data obtained by an external visual sensor, and specifically includes the following collected data: The location and maturity data of fruits and vegetables are captured by external visual sensors. The maturity information is calibrated with CSd. The maturity information CSd is obtained by collecting the color C1, size C2, and shape C3 data of fruits and vegetables in the image and then calculating the function. The location information Wz1 includes the GPS coordinates of latitude, longitude, and altitude. Fruit and vegetable type and characteristic data, based on the type characteristics LXTz of different fruits and vegetables, record their specific growth cycle and maturity characteristics; The current position and status data of the robot arm, including the coordinates Zb and direction Fx of the robot arm, and the status ZT data of the task currently being executed; The formula for the first picking information M1 is: Where ln, sin, exp represent natural logarithm, square root, sine, and exponential function respectively; α, β are adjustment parameters; Wz1=(GPS lat ,GPS lon ,GPSalt) Merge the elements of Wz1 into a scalar value. Specifically, use the average value of longitude, latitude, and altitude to represent the scalar value of Wz1. The calibration formula is as follows: Among them, Wz1 represents the precise location information of fruits and vegetables, including longitude GPS lat , latitude GPS lon and altitudeGPSalt; CSd=k1·C1+k2·C2+k3·C3 CSd represents maturity information, which is a function based on the color C1, size C2 and shape C3 of fruits and vegetables. k1, k2 and k3 are weight parameters set according to actual conditions, which are used to adjust the contribution of color, size and shape to maturity assessment. S2, using a drone equipped with a visual recognition camera to collect visual image data of the area to be picked in the air, and adjusting the wind direction generated by the rotation of the drone's rotor to reveal the fruits and vegetables blocked by branches and leaves, and generating second picking information M2 for picking mature fruits and vegetables in a large range; after obtaining the first picking information M1 and the second picking information M2, performing a comparison analysis of the mature fruits and vegetables information, and generating information data of mature fruits and vegetables blocked by branches and leaves; specifically including the following contents: Monitor the impact of the wind generated by the rotating drone rotors on the surrounding vegetation; adjust the drone's flight altitude and angle, or adjust the rotor speed to generate appropriate wind to expose the fruits and vegetables hidden by branches and leaves; While adjusting the wind direction, the visual recognition camera on the drone continues to capture images and identify fruits and vegetables exposed by the wind. The collected image data is sent to the processing center for image analysis and identification of mature fruits and vegetables. The second picking information M2 specifically includes the following collected data: The wide-area fruit and vegetable distribution data represents the wide-area distribution map of the area to be picked provided by the drone, including the group distribution QTFb and row spacing HJj of fruits and vegetables; The drone reveals the information data of fruits and vegetables blocked by branches and leaves through the wind generated by its rotor. The specific data includes the blocking position information, the characteristics of the visible part, the blocking degree, the fruit and vegetable group information, and the branch and leaf movement data. The data are calibrated in turn to form the blocking position information Wz2, the characteristics of the visible part KJBf, the blocking degree ZDCs, the fruit and vegetable group information GSQt, and the branch and leaf movement data YDSj: The blocking position information Wz2 indicates the latitude, longitude and altitude of the blocked fruits and vegetables in the farmland; The feature KJBf of the visible part represents the feature of the visible area after the fruits and vegetables are blocked. The features of the visible area include D1 color, D2 size, and D3 shape; The occlusion degree ZDCs indicates the degree to which fruits and vegetables are obscured. The fruit and vegetable group information GSQt represents the data of multiple fruits and vegetables in the area being blocked; The branch and leaf movement data YDSj represents the movement of branches and leaves caused by the wind force of the drone rotor, including the movement amplitude YDFd and frequency f of the branches and leaves; The formula for the second picking information M2 is: Among them, e x ,tanh(x),Γ(x),x y ,ln(x) represents exponential, hyperbolic tangent, gamma function, power function and natural logarithm respectively; QTFb represents the wide-area distribution of fruit and vegetable groups; HJj represents the row spacing in the field; The formula for the position information Wz2 of the obscured fruits and vegetables is: Wz2=(GPS lat2 ,GPS lon2 ,GPS alt2 ) Combine the elements of Wz2 into a scalar value. Specifically, use the average value of longitude, latitude, and altitude to represent the scalar value of Wz2. The calibration formula is as follows: Among them, Wz2 represents the precise location information of fruits and vegetables, including longitude GPS lat2 , latitude GPS lon2 and altitude GPS alt2 The characteristic KJBf formula of the visible part of fruits and vegetables is: KJBf=k4·D1+k5·D2+k6·D3 Among them, color D1, size D2, shape D3, k4, k5 and k6 are weight parameters set according to actual conditions; S3, collecting the surrounding environment information of the mature fruits and vegetables blocked by branches and leaves and generating the third picking information M3; integrating the collected data in the first picking information M1, the second picking information M2 and the third picking information M3 and analyzing and processing them to generate a picking evaluation model P, and determining the value range set of the picking evaluation model P, and planning the pre-picking path between the robot arm and the target fruit based on the value range set of the picking evaluation model P, and generating an initial planning path H1; specifically including the following contents: The third picking information M3 includes integrated meteorological data QXSj, light condition data GZTj, vegetation density ZBMd and historical picking data LSCz: The integrated meteorological data QXSj specifically collects the temperature Wd, humidity Sd, wind speed Fs, and light condition GZTj data around the mature fruits and vegetables that are blocked by branches and leaves; The light condition data GZTj include sunshine duration t1 and sun position tw; The third picking information M3 formula is: Among them, (Φ(x), log b (a)) represent the standard normal distribution function, the cube root and the logarithm of a with base b, respectively; The steps of fusing the collected data in the first picking information M1, the second picking information M2 and the third picking information M3 and analyzing and processing them to generate a picking evaluation model P, and determining a value range set of the picking evaluation model P, specifically include the following contents: Set the picking evaluation model P as: in: Wz1: represents the precise location information of fruits and vegetables; CSd: represents maturity information, which is calculated from the color C1, size C2, and shape C3 of fruits and vegetables; LXTz: represents fruit and vegetable types and characteristics; Zb, Fx, ZT: represent the coordinates, direction and current task status of the robot arm respectively; M1, M2, M3: represent the first, second and third picking information respectively; f(i) is an advanced integral function, which represents a comprehensive analysis of environmental factors; the advanced integral function f(i) is defined as YDFd and GSQt represent the movement amplitude of branches and leaves and the information of fruit and vegetable population, respectively; QTFb, HJj, Wz2, KJBf, and ZDCs represent the distribution of fruit and vegetable groups, row spacing, occlusion position information, characteristics of the visible part, and occlusion degree, respectively; S4, after obtaining the first picking information M1, the second picking information M2 and the third picking information M3, generate a branch and leaf occlusion evaluation index ZDZs, set a comparison threshold U of the branch and leaf occlusion evaluation index ZDZs, and use the comparison threshold U of the branch and leaf occlusion evaluation index ZDZs to determine whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion to pick fruits and vegetables during the picking motion planning, and generate the planned path H2 and the planned path H3 in turn; specifically including the following contents: The first picking information M1, the second picking information M2 and the third picking information M3 are obtained and a branch and leaf occlusion evaluation index ZDZs is generated, and a comparison threshold A of the branch and leaf occlusion evaluation index ZDZs is set. The threshold A of the branch and leaf occlusion evaluation index ZDZs is used to determine whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion to pick fruits and vegetables when moving; The calculation formula of the branch and leaf shading evaluation index ZDZs is as follows: Among them, φ(M1,M2,M3,t) is an integral function based on picking information, which represents the temporal dynamic change of occlusion during the picking process, and its form is: φ(M1,M2,M3,t)=λ1·M1(t)+λ2·sin(M2(t))+λ3·cos(M3(t)) ω i is the normalized weight, F i (H2, H3) is a complex information filtering function based on path planning, which represents the occlusion situation under different path planning; this function can be specifically expressed as: Among them, M1, M2, M3 represent the first, second, and third picking information respectively; t represents the time variable; T is the total duration of the picking process; A is the set threshold; λ1, λ2, λ3 are adjustment coefficients; H2 and H3 represent two different path planning; N is the number of path planning; ò is a small amount to prevent the denominator from being zero; The value range of ZDZs is set to (0, A). When ZDZs is closer to A, the movement of the fruit and vegetable picking robot arm in the occlusion environment is less affected, and the optimized planning path H2 is generated. The path planning ignores the occlusion of branches and leaves and takes the shortest path trajectory of mature fruits and vegetables, that is, the shortest path is selected to directly pass through the branch and leaf area; If the branch and leaf occlusion evaluation index ZDZs is closer to 0, it is judged that the movement of the fruit and vegetable picking robot arm in the occlusion environment is greatly affected, and it is more inclined to choose a path that avoids the occlusion, and generate the optimized planning path H3. In this way, the path planning focuses on the minimum touch points of branch and leaf occlusion, reducing the probability of the fruit and vegetable picking robot arm being stuck by branches and leaves when turning; Characters explained: e is the base of the natural logarithm; dt is the integral over time, indicating the cumulative effect of occlusion over time; φ is the integral function based on the picking information; is a summation function used to weight the information filtering results of different path planning; ω i is the normalized weight; S5. Continuously obtain data on environmental changes of the picking robot arm during the picking process, set an emergency response mechanism for the picking robot arm to escape from a stuck state, and determine whether new obstructions appear or the existing obstruction status changes to determine whether to trigger the emergency response mechanism.

2. The picking robot arm motion planning method according to claim 1, characterized in that: The value range of the picking evaluation model P is (0, +∞), and different values ​​represent different picking efficiencies and path optimization degrees; a lower picking evaluation model P value indicates a higher picking efficiency and a more optimized path planning, while a higher picking evaluation model P value means that the picking strategy or path planning needs to be improved; The pre-picking path between the robot arm and the target fruit is planned based on the value range set of the picking evaluation model P to generate an initial planning path H1, which specifically includes the following contents: Set the interval values ​​A and B of the picking evaluation model P, and A is less than B, and divide the value interval of the picking evaluation model P into a high efficiency interval (0, A), a medium efficiency interval (A, B) and a low efficiency interval (B, +∞); Among them, the high efficiency interval (0, A) represents very high picking efficiency and optimized path. The path planning principle is to respond quickly to significant changes in environmental factors f(i) and generate the initial planning path H1; The medium efficiency interval (A, B) indicates that the picking efficiency is at a medium level. The path planning principle is to set up a feedback mechanism to analyze whether the efficiency decreases due to the fruit and vegetable type characteristics LXTz or the branch and leaf movement amplitude YDFd, and make corresponding adjustments to generate the initial planning path H1; The low efficiency interval (B, +∞) indicates that the picking efficiency is at a low level. The path planning principle is to set up a deep analysis process and focus on analyzing the influence of the occlusion degree ZDCs and the occlusion position information Wz2. If the measurement results are in this interval for several consecutive times, the picking strategy or path planning is comprehensively redesigned to generate the initial planning path H1.

3. The picking robot arm motion planning method according to claim 1, characterized in that: The continuously acquiring the environmental change data of the picking robot arm during the picking process and judging whether there is a new occlusion or the existing occlusion state changes to determine whether to trigger the emergency response mechanism specifically includes the following contents: Emergency response mechanisms include: Reverse movement: The robot arm tries to move back slightly to get out of the stuck state; Occlusion analysis: Re-evaluate the occluded area to determine if additional mechanical actions are required, such as rotation or lateral movement, to avoid the obstacle; Path replanning: Dynamically replan the path based on new occlusion data to find a new optimal path.

4. A picking robot arm motion planning system, characterized in that: The system is used to execute the picking robot arm motion planning method according to any one of claims 1 to 3, comprising: The first picking information establishment module: obtains historical picking data and a botanical database of the area to be picked, and uses them to identify key information on the growth cycle and maturity characteristics of different fruits and vegetables; collects information on mature fruits and vegetables on the overall moving path of the picking robot arm in the area to be picked, and generates the first picking information M1; Second picking information establishment module: Use a drone equipped with a visual recognition camera to collect visual image data of the picking area from the air, and adjust the wind direction generated by the rotation of the drone's rotor to reveal the fruits and vegetables blocked by branches and leaves, and generate second picking information M2 for picking mature fruits and vegetables in a large range; After obtaining the first picking information M1 and the second picking information M2, compare and analyze the mature fruits and vegetables information to generate information data of mature fruits and vegetables blocked by branches and leaves; Picking evaluation model construction module: collect the surrounding environment information of mature fruits and vegetables blocked by branches and leaves and generate the third picking information M3; generate the picking evaluation model P after integrating the collected data in the first picking information M1, the second picking information M2 and the third picking information M3 and analyzing and processing them, and determine the value range set of the picking evaluation model P, plan the pre-picking path between the robot arm and the target fruit based on the value range set of the picking evaluation model P, and generate the initial planning path H1; Branch and leaf occlusion evaluation index establishment module: after obtaining the first picking information M1, the second picking information M2 and the third picking information M3, the branch and leaf occlusion evaluation index ZDZs is generated, and the comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is set. The comparison threshold U of the branch and leaf occlusion evaluation index ZDZs is used to judge whether the fruit and vegetable picking robot arm can directly ignore the branch and leaf occlusion to pick fruits and vegetables during the picking motion planning, and the planning path H2 and the planning path H3 are generated in turn; Emergency response mechanism feedback module: continuously obtains environmental change data of the picking robot arm during the picking process, sets an emergency response mechanism for the picking robot arm to escape from a stuck state, and determines whether new occlusions appear or the existing occlusion status changes to determine whether to trigger the emergency response mechanism.

Images