Fully autonomous orchard transport operation methods

Abstract

This invention discloses a fully autonomous orchard transport operation method. It acquires images and point clouds of the orchard operating environment and establishes a skeleton channel model. Based on the skeleton channel model, it determines the central guide line and passable areas between fruit tree rows, and generates a central area mask based on the central guide line. Using a preset sliding window, it scans segment by segment along the passable area and calculates the unobstructedness and congestion features of each window. The unobstructedness and congestion features are weighted and fused with the central area mask to obtain a channel heat map. The continuous area with the highest heat value in the channel heat map is selected as the optimal passageway. This invention establishes a skeleton channel model, which clearly defines the effective passage boundaries between fruit tree rows. The generated channel heat map visually presents the passage priority of each area, enabling the selection of the most suitable movement path in the irregular passage environment of the orchard.

Description

Technical Field

[0001] This invention relates to the field of autonomous handling technology, and in particular to a fully autonomous orchard handling operation method. Background Technology

[0002] In the process of modern agricultural development, the demand for automation and intelligentization of orchard handling operations is becoming increasingly urgent. Traditional orchard handling work mostly relies on manual operation, which is labor-intensive, inefficient, and unable to meet the production needs of large-scale orchards. With the development of technology, although some automated handling platforms have emerged, these devices still face many challenges in the complex orchard environment.

[0003] Existing orchard environments often present challenges due to complex terrain, irregular tree distribution, and numerous obstacles such as low-lying branches and stones between rows of trees. Current orchard transport path planning technologies, often based on single sensors (e.g., relying solely on visual images) or simple graphical models, struggle to accurately identify and handle the complexities of the orchard environment. For instance, relying solely on visual images makes it difficult to obtain three-dimensional information about the environment and accurately determine the height and distance of obstacles, leading to frequent collisions with transport equipment. Furthermore, path planning based on simple graphical models cannot adapt to the dynamic changes in the orchard environment. When determining transport paths, existing technologies often fail to comprehensively consider the actual traffic conditions of orchard passageways. For example, some methods focus only on path passability, neglecting the narrowness of the passageway and the stability of the vehicle, making it prone to collisions with obstacles when transport equipment travels between narrow rows of trees. This not only affects transport efficiency but also poses safety hazards.

[0004] Therefore, given the complex operating environment of orchards, there is an urgent need for a method that can accurately establish an orchard environment model and rationally and precisely plan the travel path of the transport platform. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of the prior art by providing a fully autonomous orchard transportation method.

[0006] The objective of this invention is achieved through the following technical solution: a fully autonomous orchard transport operation method, which acquires images and point clouds of the orchard operating environment and establishes a skeleton channel model; determines the center guide line and passable area between fruit tree rows based on the skeleton channel model; generates a center area mask based on the center guide line; uses a preset sliding window to scan segment by segment along the passable area and calculates the smoothness feature value and congestion feature value of each window; the smoothness feature value is used to represent the stability of the vehicle being able to drive in the center of the window, and the congestion feature value is used to represent the narrowness between fruit tree rows in the window; the smoothness feature value, congestion feature value and center area mask are weighted and fused to obtain a channel heat map; the continuous area with the highest heat value in the channel heat map is selected as the optimal passage corridor; the transport platform is used for path planning based on the optimal passage corridor; after the transport platform reaches the target point, the depth image of the target object is acquired and the target grab position is calculated.

[0007] As a preferred option, the current window position Smoothness feature value The calculation method is as follows:

[0008] ;

[0009] in, Current window position Effective net width, As a relaxation factor, Current window position Boundary stability coefficient, Current window position The visibility coefficient, The current heading and current window position of the transport platform The angle between the average tangents of the inner centerline;

[0010] Current window position Crowding characteristic value The calculation method is as follows:

[0011] ;

[0012] in, This is for static occupancy. It represents the time extrapolation of occupancy for a potentially moving target, used to measure the risk of future crowding out; This represents the mean normalized inverse distance from the center line to the nearest obstacles on either side within the current window position; , These are the weighting coefficients.

[0013] Preferably, when planning the path of the transport platform through the optimal passage corridor, an initial reference path is first generated in the region of the optimal passage corridor based on the target point, and the smoothness feature value and safety margin feature value of the path are calculated. The smoothness feature value is used to represent the steering continuity of the vehicle on the path, and the safety margin feature value is used to represent the distance between the two sides of the path and obstacles. Based on the reference path, the driving state of the transport platform in the future time domain is predicted to obtain the predicted stability feature value. The predicted stability feature value is used to represent the feasibility of the transport platform maintaining its trajectory in the future. The smoothness feature value, safety margin feature value and predicted stability feature value are weighted and fused to obtain a path heatmap. The target driving path is found through the path heatmap.

[0014] As a preferred method, interpretable lightweight forward roll prediction is used to calculate the prediction stability eigenvalues, as follows:

[0015] A motion model of the transport platform is generated, with a control period Δt as the time step. The state of the transport platform is extrapolated frame-by-frame within the prediction window T, including its position, heading angle, and speed. Simultaneously, the channel environment and dynamic obstacles are extrapolated. At each moment, it is determined whether all constraints are simultaneously satisfied. The total number of time steps K within the prediction window that satisfy the constraints is counted, and the percentage of time steps where constraints are satisfied is calculated. The proportion of time for constraint satisfaction The calculation formula is as follows:

[0016] ;

[0017] Where M is the total number of time steps within the prediction window, M = T / Δt;

[0018] The proportion of time during which constraints are satisfied is used as the predictive stability feature value.

[0019] As a preferred method, the smoothness feature value is calculated as follows:

[0020] First, calculate the curvature energy P. The method for calculating curvature energy P is as follows:

[0021] ;

[0022] in Let be the discrete curvature of the k-th path segment, and N be the total number of discrete path segments. For experience weights;

[0023] Then calculate the smoothness eigenvalues. The calculation formula is as follows:

[0024] ;

[0025] The norm(·) function maps the curvature energy calculation results to the range of 0 to 1;

[0026] The method for calculating the safety margin eigenvalue is as follows:

[0027] Calculate the lateral margin, occupancy suppression, and ground risk terms for each location point along the path. Use the product of these three terms as the safety value for the current location point. Normalize the safety value for this location point. Use the lowest safety value along the path as the safety margin feature value for this path.

[0028] As a preferred method, the following approach is used to find the target driving path: find the continuous segment with the highest heat value on the path heat map; if multiple breakpoints appear, select the path with the largest total heat integral and the fewest changes as the target driving path.

[0029] Preferably, when calculating the target gripping position, the geometric base feature value of the target object is first calculated, and the target object region is extracted based on the geometric base feature value and a target region mask is generated. A sliding window is used to perform a sliding window scan within the target object region, and the gripping stability feature value and deformation complexity feature value of each window are calculated. The gripping stability feature value is used to indicate the firmness of the target object being gripped at the window position, and the deformation complexity feature value is used to indicate the possibility of displacement or slippage when the target object is gripped at the window position. The gripping stability feature value, deformation complexity feature value and target region mask are weighted and fused to obtain a gripping heat map. In the gripping heat map, the region with a heat value lower than a preset threshold is selected as the stable support part of the object, and this part is used as the target gripping position of the transport platform.

[0030] Preferably, the method for calculating the geometric base eigenvalues ​​is as follows:

[0031] First, calculate the low centroid term, the normal vertical term, and the local flatness term. The low centroid term is used to quantify whether the height of the target point from the ground plane is close to the nominal height of the target object's base. The normal vertical term is used to determine the perpendicularity of the surface where the target point is located to the ground. The local flatness term is used to determine whether the height of the area around the target point is uniform. The product of the low centroid term, the normal vertical term, and the local flatness term is used as the geometric base characteristic value of the target point on the target object.

[0032] Capture stability feature values The calculation formula is as follows:

[0033] ;

[0034] in, The flattening support term at the current window position q is represented by the inverse function of the normalized residuals from the neighborhood points to the best-fit plane; For the current window position q, the envelope matching term represents the degree of matching between the opening width of the gripper on the transport platform and the local thickness of the target object; The anti-slip risk coefficient of the surface at the current window position q; The anti-eccentric load term for the current window position q is calculated by normalizing the lever arm from the window position to the target centroid.

[0035] Deformation complexity eigenvalue of the current window position q The calculation method is as follows:

[0036] ;

[0037] Among them, The largest eigenvalue, derived from the structural tensor or point cloud covariance matrix of the target region, reflects the stiffness or strength of the region along the principal direction. The larger the value, the stronger the region's extensibility along the principal direction, and the easier it is to stably bear loads.

[0038] This is the second largest eigenvalue, representing the degree of deformation in this region along this direction. If... and If the regions are close together, it indicates strong isotropy, making them prone to multi-directional deformation, which is detrimental to stable grasping.

[0039] The minimum eigenvalue reflects the region's resistance to deformation in the thickness direction. A smaller value indicates that the region is weak in the vertical direction, and may collapse or bend during gripping.

[0040] This is a regularization parameter (stability factor) used to avoid zero denominators and control instability caused by excessively large eigenvalue ratios. It is typically set to a minimum value, such as 10. -6 .

[0041] Preferably, after the transport platform grabs the target object and travels to the unloading point, it first acquires an image of a ground reference object or marker point, and adjusts the position of the transport platform to keep the transport platform aligned with the unloading point; during the unloading process of the target object, the height of the robotic arm is gradually reduced, and the attitude stability characteristic value is calculated. The attitude stability characteristic value is used to represent the degree of balance of the target object when it contacts the ground; when the attitude stability characteristic value reaches a preset threshold, the robotic arm on the transport platform releases the target object.

[0042] Preferably, the transport platform includes a tracked chassis, with two robotic arms respectively arranged on both sides of the tracked chassis. The front end of the two robotic arms is provided with a gripping module, which includes a clamping plate and a clamping driver arranged between the clamping plate and the robotic arm. Navigation modules are respectively arranged at the front and rear ends of both sides of the tracked chassis.

[0043] The beneficial effects of this invention are as follows: This invention achieves precise perception of the orchard environment by fusing dual-source data of orchard operation environment images and point clouds, and by establishing a skeleton channel model. The skeleton channel model can clearly define the effective passage boundaries between fruit tree rows, providing a precise environmental basis for subsequent path planning. By scanning the passable area segment by segment through a preset window, and simultaneously calculating the smoothness feature value and the congestion feature value, the channel quality is quantified from two dimensions: stability and space sufficiency, thus providing an effective guarantee for the safe and stable operation of the subsequent transport platform in the orchard. By weightedly fusing the dual feature values ​​with the central area mask, the generated channel heat map can intuitively present the passage priority of each area, that is, the continuous area with the highest heat value is the best passage corridor, ensuring the accuracy of the planned path and being able to select the most suitable movement path in the irregular channel environment of the orchard. Attached Figure Description

[0044] Figure 1 This is a structural diagram of the transport platform.

[0045] Figure 2 This is a path heatmap.

[0046] Figure 3 Capture heatmaps for different values.

[0047] In the diagram: 001, tracked chassis; 002, first robotic arm unit; 003, first joint motor; 004, second robotic arm unit; 005, second joint motor; 006, third robotic arm unit; 007, third joint motor; 008, clamping driver; 009, clamping plate; 010, guide rod. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0049] Those skilled in the art should understand that, in the disclosure of this invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limiting this invention.

[0050] It is understood that the term "a" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element can be one, while in another embodiment, the number of the element can be multiple, and the term "a" should not be understood as a limitation on the number.

[0051] like Figures 1 to 3 As shown, the fully autonomous orchard transport operation method acquires images and point clouds of the orchard operating environment and establishes a skeleton channel model. Based on the skeleton channel model, it determines the center guide line and passable area between fruit tree rows, and generates a center area mask based on the center guide line. Using a preset sliding window, it scans segment by segment along the passable area and calculates the smoothness feature value and congestion feature value of each window. The smoothness feature value is used to represent the stability of the vehicle being able to drive in the center of the window, and the congestion feature value is used to represent the narrowness between fruit tree rows in the window. The smoothness feature value, congestion feature value and center area mask are weighted and fused to obtain a channel heat map. The continuous area with the highest heat value in the channel heat map is selected as the optimal passage corridor. The transport platform is used for path planning based on the optimal passage corridor. After the transport platform reaches the target point, the depth image of the target object is acquired and the target grab position is calculated.

[0052] The central guide line is used to indicate the geometric center of the passable path in the orchard. After acquiring images and point clouds of the orchard's working environment, the spatial boundary feature values ​​of the tree rows are calculated, and a skeletonized passage model is constructed based on these spatial boundary feature values; wherein, the spatial boundary feature values ​​are used to represent the shape constraints of the tree rows.

[0053] In the process of orchard environmental perception, it is necessary to first acquire images, point clouds, and lighting information of the orchard's working environment, calculate the illumination balance feature value of the image, and adjust the camera's exposure and gain parameters based on this feature value. The illumination balance feature value is used to represent the overall brightness distribution of the current image. An environmental image mask is generated based on the adjustment result, and reflective or overly dark areas in the image are removed. The spatial integrity feature value of each region is calculated in combination with the point cloud density. The integrity feature value is used to indicate whether there is occlusion or missing data in the region. The environmental image mask and the spatial integrity feature value are weighted and summed to obtain a perception confidence map. Regions with confidence values ​​higher than a threshold in the confidence map are selected as effective inputs for subsequent modeling and path planning.

[0054] This invention achieves precise perception of the orchard environment by fusing dual-source data—images and point clouds—and establishing a skeleton channel model. The skeleton channel model clearly defines the effective passage boundaries between rows of fruit trees, providing a precise environmental basis for subsequent path planning. By scanning the passable area segment by segment through a preset window and simultaneously calculating unobstructed and congested feature values, the channel quality is quantified from two dimensions: stability and space sufficiency. This effectively ensures the safe and stable operation of the transport platform in the orchard. The dual feature values ​​are weighted and fused with a central area mask (which emphasizes the priority of the central area of ​​the channel, preventing equipment from approaching tree roots or low branches). The resulting channel heatmap visually presents the passage priority of each area; the continuous area with the highest heat value is the optimal passage corridor, ensuring the accuracy of the planned path and enabling the selection of the most suitable movement path in the irregular channel environment of the orchard.

[0055] Among them, the sliding window technique is an existing technology and a method for local data extraction and analysis. Its essence is to define a "window" of fixed size or dynamically adjustable size, and move this window successively along the spatial dimension (such as the length or width of the channel) or time dimension of the data. After each movement, specific calculations are performed on the local data contained in the window, and finally, local analysis results covering the entire data area are obtained.

[0056] Among them, the front window position Smoothness feature value The calculation method is as follows:

[0057] ;

[0058] in, Current window position The effective clear width is determined by the minimum passable width on both sides of the center line inside the window and the width of the transport platform; As a relaxation factor, Avoid division by zero and oversensitivity. Current window position The boundary stability coefficient measures the tree wall shaking or branch disorder by the variance of the point cloud normal on both sides. The smaller the boundary stability coefficient, the more stable the tree wall. Current window position The visibility coefficient, obtained from the local mean of the output confidence map, reflects whether the illumination, reflection, and occlusion are good. The value of the visibility coefficient ranges from 0 to 1. The current heading and current window position of the transport platform The angle between the average tangents of the inner centerline; the smaller this value, the better.

[0059] This formula unifies geometric width, surface stability, and row alignment into a one-dimensional comparable index, which facilitates subsequent weighted fusion.

[0060] The congestion feature value of the front window position measures the degree to which the current window is "occupied". The higher the value, the more congested and less convenient it is to pass through.

[0061] Current window position Crowding characteristic value The calculation method is as follows:

[0062] ;

[0063] in, Static occupancy represents the average probability of occupancy being 0 within a window. Static occupancy is based on preprocessed point cloud data, obtained through semantic segmentation or obstacle detection algorithms. The probability of occupancy for each spatial unit within the window is calculated, where 1 indicates complete occupancy and 0 indicates complete vacancy. The arithmetic mean of the occupancy probabilities for all spatial units within the window is taken. If the static occupancy is close to 1, it indicates that there are a large number of static obstacles (such as fallen branches or stacked fruit boxes) within the window; if it is close to 0, it indicates that there is no immediate occupancy in the current area and the static passage conditions are good.

[0064] This represents the time extrapolation of the occupancy of a potentially moving target, used to measure the risk of future encroachment. This value is calculated based on extrapolating the displacement vector of the most recent two or three frames by 1 to 2 seconds. This represents the mean normalized inverse distance from the centerline to the nearest obstacles on both sides within the current window position. It is used to measure the distance risk between the transport platform and obstacles on both sides of the passage, and reflects the actual passable width redundancy of the passage. The closer a value is to 1, the closer the vehicle is to the boundaries on both sides (e.g., excessive branches and leaves encroaching on the passage), and the lower the traffic redundancy; the closer a value is to 0, the more sufficient the boundary distance is, and there is no risk of approaching the boundary.

[0065] , These are the weighting coefficients. The value ranges from 0.6 to 1.2, and the weighting coefficient is... The value range is 0.3 to 0.8.

[0066] When planning the path of the transport platform using the optimal passage corridor, an initial reference path is first generated in the region of the optimal passage corridor based on the target point, and the smoothness feature value and safety margin feature value of the path are calculated. The smoothness feature value is used to represent the steering continuity of the vehicle on the path, and the safety margin feature value is used to represent the distance between the sides of the path and obstacles. Based on the reference path, the driving state of the transport platform in the future time domain is predicted to obtain the predicted stability feature value. The predicted stability feature value is used to represent the feasibility of the transport platform maintaining its trajectory in the future. The smoothness feature value, safety margin feature value and predicted stability feature value are weighted and fused to obtain the path heatmap. The target driving path is found through the path heatmap.

[0067] This invention quantifies the "turning continuity" of a path using smoothness feature values. Smoothness feature values ​​primarily reflect whether the trajectory turning is continuous, and whether there are sharp turns or broken lines, directly reflecting the "operational difficulty" of the equipment. Incorporating smoothness feature values ​​into path selection prioritizes reference paths with "few turns and gentle trajectories," ensuring the transport platform travels at a stable speed, reducing mechanical wear and energy consumption, and protecting the integrity of the transported goods, especially suitable for scenarios involving the transport of easily damaged fresh fruit. The invention quantifies the "safety buffer space" of the path using "safety margin feature values." Incorporating this feature ensures that the final path is not only "passable" but also has sufficient "safety redundancy," thereby protecting orchard assets (fruit trees, fruit) and equipment. Finally, the invention achieves "dynamic risk avoidance" through "predictive stability feature values." This feature allows path planning to dynamically adapt to the future, preventing equipment from traveling to areas that "appear safe now but will be impassable in the future" (such as a sudden narrowing of the passage ahead or slippery ground causing trajectory deviation), significantly improving operational safety in complex environments.

[0068] The smoothness feature value is calculated as follows:

[0069] First, calculate the curvature energy P. The method for calculating curvature energy P is as follows:

[0070] ;

[0071] in Let be the discrete curvature of the k-th path segment (which can be calculated through three-point circle fitting), and N be the total number of discrete path segments. For experience weight, ;

[0072] Then calculate the smoothness eigenvalues. The calculation formula is as follows:

[0073] ;

[0074] The norm(·) function maps the calculated curvature energy to the range of 0 to 1.

[0075] The closer the calculated value of the smoothness feature is to 1, the smoother the path.

[0076] The intuitive meaning of the safety margin characteristic value is "how far away from east and west" on both sides of the trajectory, and the degree of safety from the risk source (dike, fall boundary).

[0077] The method for calculating the safety margin eigenvalue is as follows:

[0078] Calculate the lateral margin, occupancy suppression, and ground risk terms for each location point along the path. Use the product of these three terms as the safety value for the current location point. Normalize the safety value for this location point. Use the lowest safety value along the path as the safety margin feature value for this path.

[0079] The formula for calculating the lateral margin term is as follows:

[0080] The formula for calculating the occupancy suppression term is as follows:

[0081] ;

[0082] The formula for calculating ground risk items is as follows:

[0083] ;

[0084] This represents the normal distance from the path point to the nearest obstacle. This is the minimum distance between the transport platform and the obstacle. This value can be preset in advance and ranges from 0.1 to 0.2 m. For the desired margin, users can preset it in advance, with a value range of 0.3 to 0.5m. This represents the occupancy probability. A value of 1 indicates full occupancy, while a value of 0 indicates complete idleness.

[0085] The probability of ground risk can be defined by actual conditions such as ground roughness, humidity, and distance from the impact edge, and its value ranges from 0 to 1.

[0086] This invention employs interpretable, lightweight forward roll prediction to calculate the prediction stability eigenvalues. The specific method is as follows:

[0087] A motion model of the transport platform is generated, with a control period Δt as the time step. The platform's state is extrapolated frame-by-frame within the prediction window T, including its position, heading angle, and speed. Simultaneously, the channel environment and dynamic obstacles are extrapolated. At each moment, it is determined whether all constraints are simultaneously satisfied, including maximum turning angle / angular velocity limits, minimum lateral distance limits, maximum longitudinal and lateral acceleration limits, and ground adhesion limits. The number K of all time steps within the prediction window that satisfy the constraints is counted, and the percentage of time steps where constraints are satisfied is calculated. The proportion of time for constraint satisfaction The calculation formula is as follows:

[0088] ;

[0089] Where M is the total number of time steps within the prediction window, M = T / Δt;

[0090] The proportion of time during which constraints are satisfied is used as the predictive stability feature value.

[0091] The method for finding the target driving route is as follows: Find the continuous segment with the highest heat value on the route heatmap; if multiple breakpoints appear, select the route with the largest total heat integral and the fewest changes as the target driving route. The route heatmap is shown below. Figure 2 As shown.

[0092] When calculating the target gripping position, the geometric base feature value of the target object is first calculated. Based on the geometric base feature value, the target object region is extracted and a target region mask is generated. A sliding window is used to perform a sliding window scan within the target object region, and the gripping stability feature value and deformation complexity feature value of each window are calculated. The gripping stability feature value is used to indicate the firmness of the target object being gripped at the window position, and the deformation complexity feature value is used to indicate the possibility of displacement or slippage when the target object is gripped at the window position. The gripping stability feature value, deformation complexity feature value and target region mask are weighted and fused to obtain a gripping heat map. In the gripping heat map, the region with a heat value lower than a preset threshold is selected as the stable support part of the object, and this part is used as the target gripping position of the transport platform.

[0093] In the depth map or point cloud, select regions that meet the criteria of "low height + normal close to vertical + small angle with the ground" as candidate base pixels.

[0094] Specifically, the method for calculating the eigenvalues ​​of the geometric basis is as follows:

[0095] First, calculate the low centroid term, the normal vertical term, and the local flatness term. The low centroid term is used to quantify whether the height of the target point from the ground plane is close to the nominal height of the target object's base. The normal vertical term is used to determine the perpendicularity of the surface where the target point is located to the ground. The local flatness term is used to determine whether the height of the area around the target point is uniform. The product of the low centroid term, the normal vertical term, and the local flatness term is used as the geometric base characteristic value of the target point on the target object.

[0096] The calculation method for the low centroid term is as follows:

[0097] ;

[0098] The calculation method for the vertical normal term is as follows:

[0099] ;

[0100] The calculation method for the vertical normal term is as follows:

[0101] ;

[0102] in The height of the candidate base pixel from the ground. The nominal height of the base of the target object (take 0 if unknown). The normal of the candidate basal pixel. The height undulation of the neighborhood of the candidate base pixel. This represents the flatness coefficient of the base region in the vertical direction. It measures the flatness of the base region in the vertical direction. The smaller the value, the more stable the region is, and the more suitable it is as a gripping support point for a robotic claw. It can be defined using the local root mean square error of the depth map in the Z direction or the deviation between the centroid height of the base point set and the average height of its neighborhood.

[0103] After thresholding the geometric base feature values, a connected component filter is performed using shape priors (consistency of rectangle / cuboid edge) to obtain a region mask of "target-non-ground-near base". This generated mask naturally avoids highly reflective upper surfaces or backgrounds. A sliding window scan is performed within the target region using a preset window size, and the grasping stability feature values ​​are calculated for each region.

[0104] Capture stability feature values The calculation formula is as follows:

[0105] ;

[0106] in, The flatness support term at the current window position q quantifies the flatness of the local surface at the window position. It is represented by an inverse function of the normalized residuals from the neighborhood points to the best-fit plane. The higher the value, the flatter the support surface and the more uniform the force during clamping. .

[0107] The envelope matching term for the current window position q represents the degree of matching between the opening width of the gripper on the transport platform and the local thickness of the target object; the higher the value, the better the gripper opening matches the target structure and the more effectively it can wrap the target. .

[0108] is the anti-slip risk coefficient of the surface at the current window position q; it is estimated based on the surface roughness and material properties at the window position, and the higher the value, the greater the risk of slippage after clamping.

[0109] This is the anti-eccentric load term for the current window position q, calculated by normalizing the lever arm from the window position to the target's center of mass; a higher value indicates a more severe eccentric load on the target after clamping.

[0110] The final calculated gripping stability characteristic value is as follows: a value close to 1 indicates that the support at the window position is flat, the claw mouth is adapted, and it is resistant to slipping and off-center loading, making it an ideal gripping point; a value close to 0 indicates that there is a risk of gripping instability.

[0111] Deformation complexity eigenvalue of the current window position q The calculation method is as follows:

[0112] ;

[0113] Among them, The largest eigenvalue, derived from the structural tensor or point cloud covariance matrix of the target region, reflects the stiffness or strength of the region along the principal direction. The larger the value, the stronger the region's extensibility along the principal direction, and the easier it is to stably bear loads.

[0114] This is the second largest eigenvalue, representing the degree of deformation in this region along this direction. If... and If the regions are close together, it indicates strong isotropy, making them prone to multi-directional deformation, which is detrimental to stable grasping.

[0115] The minimum eigenvalue reflects the region's resistance to deformation in the thickness direction. A smaller value indicates that the region is weak in the vertical direction, and may collapse or bend during gripping.

[0116] This is a regularization parameter (stability factor) used to avoid zero denominators and control instability caused by excessively large eigenvalue ratios. It is typically set to a minimum value, such as 10. -6 .

[0117] If the surface of the target object is unidirectionally undulating like a thin sheet and exhibits significant third-dimensional collapse... A high value indicates that the surface is easily deformed and is not suitable for use as a stable gripping surface.

[0118] Heatmaps of different capture values ​​are attached. Figure 3 As shown.

[0119] After the transport platform grabs the target object and travels to the unloading point, it first acquires images of ground reference objects or markers, and adjusts the position of the transport platform to keep it aligned with the unloading point. During the unloading process, the height of the robotic arm is gradually reduced, and the attitude stability characteristic value is calculated. The attitude stability characteristic value is used to represent the degree of balance of the target object when it contacts the ground. When the attitude stability characteristic value reaches a preset threshold, the robotic arm on the transport platform releases the target object.

[0120] Among them, attitude stability eigenvalues The calculation method is as follows:

[0121] ;

[0122] In the formula, The support margin term represents the minimum distance projected from the center of mass of the target object onto the boundary of the supporting polygon. Its distance to the center of the polygon The ratio of .

[0123] For the contact area term, The actual contact area between the target object and the ground is calculated from the connected components of the contact patch in the depth image; This is the reference contact area of ​​the bottom surface of the target object, which is the theoretical maximum contact area when the target object is placed flat. The attenuation constant of micro-vibration after the target object is placed is estimated from vibration data read by the IMU or force sensor on the wrist of the robotic arm of the handling platform. As an empirical measure, the smaller the value, the faster the vibration decays, and the more stable the device is.

[0124] like Figure 1 As shown, the transport platform includes a tracked chassis 001, with two robotic arms on each side of the tracked chassis 001. The front ends of the two robotic arms are equipped with gripping modules, which include a clamping plate 009 and a clamping driver 008 between the clamping plate 009 and the robotic arm. Navigation modules are provided at the front and rear ends of both sides of the tracked chassis 001.

[0125] The robotic arm includes a first robotic arm unit 002, a second robotic arm unit 004, a third robotic arm unit 006, a first joint motor 003, a second joint motor 005, and a third joint motor 007. One end of the first robotic arm unit 002 is connected to the first joint motor 003, which is fixedly mounted on one side of the tracked chassis 001. The second joint motor 005 is located between the other end of the first robotic arm unit 002 and one end of the second robotic arm unit 004, and the third joint motor 007 is located between the other end of the second robotic arm unit 004 and one end of the third robotic arm unit 006. A gripping module is located at the other end of the third robotic arm unit 006. A guide rod 010 is provided on the clamping plate 009, and the direction of the guide rod 010 is consistent with the direction of movement of the clamping plate 009. The clamping plate 009 is moved by a clamping driver 008 to clamp the target object.

[0126] In this invention, the clamping actuator 008 is an electric cylinder. Driven by a joint motor, the three clamping arm units can rotate relative to each other to achieve flexible clamping of the target object.

[0127] This invention is not limited to the preferred embodiments described above. Anyone can derive other products in various forms under the guidance of this invention. However, regardless of any changes in shape or structure, any technical solution that is the same as or similar to this application falls within the protection scope of this invention.

Claims

1. A fully autonomous orchard handling method, characterized by, An image and a point cloud of an orchard working environment are acquired, and a skeleton passage model is established, a center guide line and a passable area between rows of fruit trees are determined based on the skeleton passage model, and a center area mask is generated based on the center guide line; A preset sliding window is used to scan the passable area in segments, and a passability feature value and a congestion feature value of each window are calculated; The passability feature value is used to represent the stability of the vehicle driving in the center of the window, and the congestion feature value is used to represent the narrowness of the rows of fruit trees in the window; the passability feature value and the congestion feature value are weighted and fused with the center area mask to obtain a passage heat map, and a continuous area with the highest heat value in the passage heat map is selected as the best traffic corridor; The carrying platform is path planned through the best traffic corridor; after the carrying platform reaches the target point, a depth image of the target object is acquired, and a target grabbing position is calculated; The center guide line is used to represent the geometric center position of the passable path in the orchard; after the image and the point cloud of the orchard working environment are acquired, a spatial boundary feature value of the rows of fruit trees is calculated, and a skeleton passage model is constructed based on the spatial boundary feature value; the spatial boundary feature value is used to represent the shape constraint of the rows of fruit trees; Current window position The patency characteristic value of the The calculation method is as follows:​ ； wherein, is the effective net width of the current window position , is the relaxation factor, is the boundary stability coefficient of the current window position , is the visibility coefficient of the current window position , is the angle between the current heading of the transport platform and the average tangential direction of the center line within the current window position ; Current window position The congestion characteristic value of the current window position The calculation method is as follows: ； wherein, is the static occupancy, represents a time extrapolated occupancy for a potentially moving target, used to measure future encroachment risk; represents the mean of the normalized inverse distance of the centerline to the closest obstacle on both sides within the current window position; , is a weight coefficient.

2. The fully autonomous orchard handling method according to claim 1, characterized in that, When the carrying platform is path planned through the best traffic corridor, an initial reference path is generated in the area of the best traffic corridor according to the target point, and a smoothness feature value and a safety margin feature value of the path are calculated; The smoothness feature value is used to represent the steering continuity of the vehicle driving on the path, and the safety margin feature value is used to represent the distance between the path and the obstacles on both sides; The driving state of the carrying platform in a future time domain is predicted based on the reference path to obtain a predicted stability feature value; the predicted stability feature value is used to represent the feasibility of the carrying platform keeping the trajectory for a period of time in the future; the smoothness feature value, the safety margin feature value and the predicted stability feature value are weighted and fused to obtain a path heat map; and a target driving path is found through the path heat map; The calculation method of the smoothness feature value is as follows: First, the curvature energy P is calculated, and the calculation method of the curvature energy P is as follows: ； wherein is the discrete curvature of the kth path segment in the path, N is the total number of discrete path segments, is the empirical weight; Then the smoothness eigenvalue is calculated with the following formula: ； The norm(·) function maps the calculation result of the curvature energy to the range of 0 to 1; The calculation method of the safety margin feature value is as follows: The lateral margin term, the occupation suppression term and the ground risk term of each position point on the path are calculated respectively, and the product of the three terms is taken as the safety value of the current position point; And the safety value of the position point is normalized; The lowest safety value in the path is taken as the safety margin feature value of the path.

3. The fully autonomous orchard haul work method according to claim 2, characterized in that, The predicted stability feature value is calculated by using an interpretable lightweight forward-looking rolling prediction, and the specific method is as follows: A motion model of the carrying platform is generated, and a state of the carrying platform is deduced frame by frame in a prediction window T with a control period Δt as a time step, the state including a position, a heading angle and a running speed of the carrying platform; while the state of the carrying platform is deduced, an environment and a dynamic obstacle of a channel are extrapolated synchronously; whether all constraint conditions are satisfied simultaneously at each time is determined; a time step number K of all time steps satisfying the constraint conditions in the prediction window is counted, and a time ratio of constraint satisfaction is calculated , and the calculation formula of the time ratio of constraint satisfaction is as follows: ； Wherein, M is the total number of time steps in the prediction window, M=T / Δt; The time proportion of constraint satisfaction is taken as the predicted stability feature value.

4. The fully autonomous orchard haul job method of claim 1, wherein, The method of finding the target driving path is as follows: a continuous segment with the highest heat value is found on the path heat map; if multiple breakpoints occur, a path with the maximum total heat integral and the least number of changes is selected as the target driving path.

5. The fully autonomous orchard haul job method of claim 1, wherein, In the calculation of the target grabbing position, the geometric base feature value of the target object is calculated first, and the target object area is extracted based on the geometric base feature value and a target area mask is generated; a sliding window is used for sliding window scanning in the target object area, and the grabbing stability feature value and the deformation complexity feature value of each window are calculated, the grabbing stability feature value is used to represent the firmness of the window position clamping the target object, and the deformation complexity feature value is used to represent the possibility of deviation or slipping of the window position clamping the target object; the grabbing stability feature value, the deformation complexity feature value and the target area mask are weighted and fused to obtain a grabbing heat map, and the area with a heat value lower than a preset threshold in the grabbing heat map is selected as the stable support part of the object, and the part is taken as the target grabbing position of the carrying platform; the calculation method of the geometric base feature value is as follows: First, the low gravity item, the normal vertical item and the local flat item are calculated; the low gravity item is used to quantify whether the height of the target point to the ground plane is close to the nominal height of the target object base; the normal vertical item is used to judge the verticality of the surface where the target point is located and the ground; the local flat item is used to judge whether the height of the area around the target point is uniform; the product of the low gravity item, the normal vertical item and the local flat item is taken as the geometric base feature value of the target point on the target object; Grip stability characteristic value The formula for calculating the grip stability characteristic value is as follows: ； wherein, is a flatness support term for the current window position q, represented by the inverse function of the residual of the neighborhood points to the best-fit plane normalized by the thickness of the target object; is an envelope match term for the current window position q, representing the match between the opening width of the gripper on the handling platform and the local thickness of the target object; is a slip risk coefficient for the surface at the current window position q; is an anti-tilt term for the current window position q, calculated by normalizing the force arm of the window position to the target centroid; a morphing complexity feature value of a current window position q The calculation method is as follows: ； wherein, is the largest eigenvalue; is the second largest eigenvalue; is the smallest eigenvalue; is a regularization parameter.

6. The fully autonomous orchard haul job method of claim 1, wherein, After the carrying platform grabs the target object and drives to the unloading point, the image of the ground reference or landmark is acquired first, and the position of the carrying platform is adjusted to keep the carrying platform aligned with the unloading point; in the process of unloading the target object, the height of the mechanical arm is gradually lowered, and the posture stability feature value is calculated, which is used to represent the balance degree when the target object contacts the ground; when the posture stability feature value reaches a preset threshold, the mechanical arm on the carrying platform releases the target object.

7. The fully autonomous orchard haul job method of claim 1, wherein, The carrying platform comprises a tracked chassis, two mechanical arms are arranged on the two sides of the tracked chassis respectively, a clamping module is arranged at the front end of the two mechanical arms, the clamping module comprises a clamping plate and a clamping driver arranged between the clamping plate and the mechanical arm; navigation modules are arranged at the front end and the rear end of the two sides of the tracked chassis respectively.

Images