Visual guidance precision operation method and device of pineapple seedling clamping mechanical arm

Abstract

The present application provides a visual guidance precision work method and device for pineapple seedling clamping mechanical arm, the technical field of visual control of agricultural robot, the present application firstly extracts the skeleton curve and the contour width curve from the pineapple seedling image, generates the curvature extreme point set and the width extreme point set, and merges to obtain the candidate clamping point set; then a time window is constructed, each candidate clamping point in each frame image in the window is scored from three dimensions of geometric stability, image reliability and work rationality; then the time sequence stability score and the future sustainability score are calculated based on the comprehensive score sequence; finally, the current frame comprehensive score, the time sequence stability score and the future sustainability score are weighted and fused, the candidate point with the highest final score is selected as the optimal clamping point, and the mechanical arm is controlled to perform clamping work. The present application effectively improves the accuracy of clamping point selection and work stability through multi-dimensional space-time feature fusion.

Description

Technical Field

[0001] This invention relates to the field of agricultural robot vision control technology, specifically to a vision-guided precision operation method and device for a pineapple seedling clamping robotic arm. Background Technology

[0002] In the field of agricultural robotics, using vision-guided technology to control robotic arms for grasping and holding crops has become a research hotspot. Existing technologies, such as the Chinese patent application CN117754568A entitled "Visual Guidance Precision Operation Method, Terminal, and Computer Storage Medium for Robotic Arms," ​​disclose a visual guidance precision operation method for robotic arms. This method compensates for guidance errors by acquiring robotic arm parameters and a compensation matrix, achieving precise transformation of the target from the camera coordinate system to the robotic arm base coordinate system. However, this solution mainly addresses the coordinate transformation accuracy problem, assuming the target has been identified, and does not involve the extraction and analysis of the target's morphological features.

[0003] For flexible crops like pineapple seedlings, their irregular shapes, including bending, width variations, and leaf shading, result in significant differences in clamping effectiveness at different locations. Relying solely on single-frame image coordinate transformation cannot solve the problem of "where to select the clamping point": too close to the base may damage the growth point, while too far from the base results in an unstable clamping due to the thin seedling, and clamping at points of severe bending or abrupt width changes is prone to slippage. Furthermore, changes in field lighting and seedling movement can render single-frame image information unreliable, and current technologies lack the utilization of temporal information, making it difficult to determine the stability and future trends of candidate clamping points.

[0004] Therefore, how to dynamically and accurately select the optimal clamping point from pineapple seedlings with diverse shapes, and ensure the stability and reliability of clamping in complex environments, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a visually guided precision operation method and device for a pineapple seedling clamping robotic arm, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The vision-guided precision operation method for a pineapple seedling gripper robotic arm includes the following steps: Step 1: Obtain the pineapple seedling image, perform target segmentation on the pineapple seedling image to obtain the pineapple seedling region, and extract the skeleton curve and contour width curve based on the pineapple seedling region. Calculate the local maxima of curvature based on the skeleton curve to form a set of curvature extrema points. Step 2: Calculate the local maxima and local minima based on the contour width curve to form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as the candidate clamping point set; Step 3: Construct a time window containing the current frame and several frames before the current frame. For each candidate clamping point in each frame within the time window, extract feature values ​​on multiple preset evaluation dimensions and convert the feature values ​​into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame. Step 4: Based on the comprehensive scores of each candidate clamping point in each frame image within the time window, construct a comprehensive score sequence corresponding to each candidate clamping point. Based on the comprehensive score sequence, calculate the temporal stability score of each candidate clamping point, and calculate the future sustainability score of each candidate clamping point according to the changing trend of the comprehensive score sequence. Step 5: Perform a weighted fusion of the comprehensive score, temporal stability score, and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point.

[0007] Furthermore, the skeleton curve and contour width curve of the pineapple seedling are extracted from the pineapple seedling area. The specific steps are as follows: Morphological skeletonization was performed on the binary mask image of the pineapple seedling area. An iterative erosion algorithm was used until a connected curve with a single pixel width was obtained, which was then used as the skeleton curve of the pineapple seedling. Identify the two endpoints of the skeleton curve. Based on the position of the endpoints in the binary mask, the endpoint with the larger ordinate is taken as the base endpoint, and the endpoint with the smaller ordinate is taken as the top endpoint. The order of the skeleton curve from the base endpoint to the top endpoint is used as the traversal order of the skeleton curve. Each curve position is traversed in turn, the tangent direction at each curve position is calculated, and the normal direction perpendicular to the tangent direction is determined. Extending along the normal direction to both sides in the binary mask image, measure the pixel distance between one boundary and the other boundary of the pineapple buying area at the corresponding curve position, and use it as the width of the seedling at the curve position; The seedling widths at each curve position are arranged according to the traversal order along the skeleton curve to obtain the outline width curve.

[0008] Furthermore, the curvature extrema on the skeleton curve and the width extrema on the contour width curve are calculated as candidate clamping points. The specific steps are as follows: Following the traversal order of the skeleton curve, each pixel within the skeleton curve is traversed sequentially. Based on the current pixel and its adjacent pixels, a local tangent direction is fitted. The curvature value at the current pixel is calculated based on the change in direction of the adjacent local tangent directions. Pixels with curvature values ​​greater than the curvature values ​​of the adjacent pixels on both sides are selected as local maxima of curvature. The set of curvature extrema points is formed by the local maxima of curvature. Iterate through each pixel point along the width curve of the contour and obtain the width value at each pixel point. Select the pixel point whose width value is greater than the width values ​​of the adjacent pixels on both sides as the local maximum value point of the width, and select the pixel point whose width value is less than the width values ​​of the adjacent pixels on both sides as the local minimum value point of the width. Merge the local maximum value point of the width and the local minimum value point of the width to form a set of width extreme value points. The set of curvature extrema points and the set of width extrema points are combined into a union. For adjacent points in the union whose Euclidean distance is less than a preset distance threshold, points with larger corresponding curvature values ​​or width extrema amplitudes are merged and deduplicated to obtain a set of candidate clamping points.

[0009] Furthermore, the preset evaluation dimensions include: The multiple preset evaluation dimensions include geometric stability dimension, image reliability dimension, and task rationality dimension; In the geometric stability evaluation dimension, the geometric stability dimension is determined by the curvature change rate and the contour width change rate of the candidate clamping point. A first local neighborhood is formed by a preset number of adjacent pixels before and after the candidate clamping point on the skeleton curve. The average value of the curvature value difference of adjacent pixels in the first local neighborhood is calculated as the curvature change rate, and the average value of the body width difference of adjacent pixels in the first local neighborhood is calculated as the contour width change rate. In the image reliability dimension, the image reliability dimension is determined by the image edge density of the local area where the candidate clamping point is located. A second local neighborhood of a preset size is constructed with the candidate clamping point as the center. Edge detection is performed on the second local neighborhood, and the ratio of the number of edge pixels to the total number of pixels in the second local neighborhood is used as the image edge density. In the dimension of operational rationality, the operational rationality dimension is determined by the proportion of the arc length distance from the candidate clamping point along the skeleton curve to the base endpoint of the pineapple seedling to the total arc length of the pineapple seedling skeleton curve. The arc length distance from the candidate clamping point along the skeleton curve to the base endpoint of the pineapple seedling is calculated, and the ratio of the arc length distance to the total arc length of the pineapple seedling skeleton curve is used as the distance proportion.

[0010] Furthermore, for each candidate gripping point, its feature values ​​on the aforementioned preset evaluation dimension are extracted. The specific steps are as follows: For the geometric stability dimension, the curvature value and contour width value at the candidate clamping point are extracted, and the curvature difference and width difference between the candidate clamping point and the previous and next skeleton points on the skeleton curve are calculated respectively. The average value of the curvature difference is used as the curvature change rate, and the average value of the width difference is used as the width change rate. The curvature value, contour width value, curvature change rate, and width change rate are used as the feature values ​​of the geometric stability dimension. For the image reliability dimension, a local image block of a preset size is extracted with the candidate clamping point as the center. Edge detection is performed on the local image block, the number of edge pixels detected is counted, and the ratio between the number of edge pixels and the total number of pixels in the local image block is calculated. The ratio is used as the feature value of the image reliability dimension. For the operational rationality dimension, the arc length distance from the candidate clamping point to the base endpoint is calculated along the skeleton curve, and the total arc length of the skeleton curve is calculated. The ratio of the arc length distance to the total arc length is used as the feature value of the operational rationality dimension.

[0011] Furthermore, based on the scoring rules, the feature values ​​of each dimension are converted into corresponding dimensional scores, and the geometric stability dimensional score of each candidate point in this frame is calculated. The specific steps are as follows: For the geometric stability dimension, a first ideal range for the rate of curvature change and a second ideal range for the rate of width change are preset respectively. When the rate of curvature change of the candidate clamping point is within the first ideal range, the preset first basic score is used as the rate of curvature change score. When the rate of curvature change of the candidate clamping point is outside the first ideal interval, the rate of curvature change score is reduced according to the linear decay rule based on the distance between the rate of curvature change of the candidate clamping point and the boundary of the first ideal interval. When the width change rate of the candidate clamping point is within the second ideal range, the preset second basic score will be used as the width change rate score. When the width change rate of the candidate clamping point is outside the second ideal interval, the width change rate score is reduced according to the linear decay rule based on the distance between the width change rate of the candidate clamping point and the boundary of the second ideal interval. The curvature change rate score and width change rate score are weighted and summed according to the preset first and second weights to obtain the geometric stability dimension score.

[0012] Furthermore, based on the scoring rules, the feature values ​​of each dimension are converted into corresponding dimension scores, and the scores for each candidate point in the image reliability dimension and job rationality dimension of this frame are calculated. The specific steps are as follows: For the image reliability dimension, a preset upper limit threshold for edge density is set. When the edge density of the candidate clamping point is less than the upper limit threshold, a linear mapping is performed according to the difference between the edge density and the upper limit threshold to obtain the image reliability dimension score. When the edge density of the candidate clamping point is greater than or equal to the upper limit threshold, the image reliability dimension score is set to zero. For the task rationality dimension, the optimal value of distance ratio and the width parameter of Gaussian function are preset, and the distance ratio of candidate clamping points is input into the preset Gaussian function to obtain the task rationality dimension score; The geometric stability dimension score, image reliability dimension score, and operation rationality dimension score are weighted and summed according to the preset comprehensive weights to obtain the comprehensive score of the candidate gripping point in the current frame image.

[0013] Furthermore, based on the comprehensive score sequence, the temporal stability score of each candidate clamping point is calculated, and based on the changing trend of the comprehensive score sequence, the future sustainability score of each candidate clamping point is calculated. The specific steps are as follows: For each candidate gripping point, the comprehensive score of the candidate gripping point in multiple consecutive frames within a preset time window is extracted and arranged in chronological order of the frames to form a corresponding comprehensive score sequence. For a comprehensive score sequence, the variance of all comprehensive scores in the sequence is calculated, and the variance is matched with multiple preset variance intervals. The time series stability score corresponding to the variance interval that matches the variance is used as the time series stability score of the candidate clamping point. For the comprehensive score sequence, the least squares method is used to fit a linear trend line of the comprehensive score changing over time, the slope of the linear trend line is obtained, and the slope is matched with multiple preset slope intervals. The future sustainability score corresponding to the slope interval that matches the slope is used as the future sustainability score of the candidate clamping point.

[0014] Furthermore, the comprehensive score, temporal stability score, and future sustainability score of each candidate point in the last frame of the time window are weighted and fused. The specific steps are as follows: The temporal stability score of each candidate clamping point is normalized to obtain a normalized temporal stability value, and the normalized temporal stability value is used as a discount factor. The first intermediate score is obtained by multiplying the comprehensive score of each candidate clamping point in the last frame image by the corresponding discount factor; The future sustainability score of each candidate clamping point is normalized to obtain a future sustainability normalized value, and the future sustainability normalized value is used as a gain factor. The first intermediate score is multiplied by the corresponding gain factor to obtain the final comprehensive score of the candidate clamping point.

[0015] The present invention also provides a vision-guided precision operation device for a pineapple seedling clamping robotic arm, the device being used to perform the aforementioned vision-guided precision operation method for the pineapple seedling clamping robotic arm, comprising: The feature extraction module is used to acquire pineapple seedling images, perform target segmentation on the pineapple seedling images to obtain pineapple seedling regions, extract skeleton curves and contour width curves based on the pineapple seedling regions, and calculate local maxima of curvature based on the skeleton curves to form a set of curvature extrema points. The candidate point generation module is used to calculate local maxima and local minima based on the contour width curve, form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as a set of candidate clamping points. The scoring calculation module is used to construct a time window containing the current frame and several frames of images before the current frame. For each candidate clamping point in each frame of the time window, feature values ​​on multiple preset evaluation dimensions are extracted, and the feature values ​​are converted into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame of the image. The temporal evaluation module is used to construct a comprehensive score sequence corresponding to each candidate clamping point based on the comprehensive score of each candidate clamping point in each frame image within the time window, calculate the temporal stability score of each candidate clamping point based on the comprehensive score sequence, and calculate the future sustainability score of each candidate clamping point based on the changing trend of the comprehensive score sequence. The weighted fusion module is used to perform weighted fusion of the comprehensive score, temporal stability score and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and to determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point.

[0016] Compared with the prior art, the beneficial effects of the present invention are: This invention generates a set of candidate clamping points based on the curvature extrema of the skeleton curve and the width extrema of the contour width curve, integrating the curvature features and width variation features of the geometric shape. Compared with traditional methods based solely on curvature or fixed positions, it can more comprehensively identify suitable parts of the seedling body for clamping, thus improving the effectiveness of the candidate points. This invention also constructs a time window to comprehensively score each candidate clamping point from three dimensions: geometric stability, image reliability, and operational rationality. It combines the temporal stability score and the future sustainability score for weighted fusion, which effectively overcomes the defects of single-frame images being susceptible to noise, lighting changes, and seedling shaking. It prioritizes clamping points with stable or continuously improving quality, significantly improving the success rate and robustness of clamping operations. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 This is a graph showing the overall score change trend of each candidate clamping point in this invention within a time window; Figure 3 This is a system framework diagram of the device in this invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0019] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] Example: Please see Figure 1-3 The present invention provides a technical solution: The vision-guided precision operation method for a pineapple seedling gripper robotic arm includes the following steps: Step 1: Obtain the pineapple seedling image, perform target segmentation on the pineapple seedling image to obtain the pineapple seedling region, and extract the skeleton curve and contour width curve based on the pineapple seedling region. Calculate the local maxima of curvature based on the skeleton curve to form a set of curvature extrema points.

[0021] In this embodiment, color images of the pineapple seedlings are acquired using an RGB industrial camera mounted at the end of the pineapple seedling gripping robotic arm. During acquisition, the camera's optical axis is at approximately a 45° angle to the ground. A 60° angle should be used to ensure the pineapple seedling is completely centered in the image. To ensure the stability of subsequent processing, the lighting conditions during image acquisition should be as uniform as possible, avoiding strong backlighting or direct shadows covering the main stem of the seedling.

[0022] After acquiring the original color image, a deep learning-based semantic segmentation network is used to segment the pineapple seedling region in the image. In this embodiment, a U-Net network structure is used, with an input image size of 512×512 pixels and an output of a single-channel probability map of the same size. The probability value represents the confidence that a pixel belongs to the pineapple seedling region. The probability map is binarized with a threshold of 0.5 to obtain a binary mask image, where a pixel value of 1 indicates that the location belongs to the pineapple seedling region, and a pixel value of 0 indicates the background region. As an alternative, in scenarios with limited computing resources, a threshold segmentation method based on the HSV color space can also be used, which segments the pineapple seedling samples by statistically analyzing the hue, saturation, and brightness distribution range.

[0023] Morphological skeletonization is performed on the binary mask image. Specifically, an iterative erosion algorithm is used: boundary pixels of the seedling region are repeatedly removed while maintaining the connectivity of the seedling region, until no more pixels can be deleted. The resulting connected curve, with a single pixel width, serves as the skeleton curve of the pineapple seedling. The skeleton curve is represented as a set of ordered pixels.

[0024] To identify the two endpoints in the set of ordered pixels of the skeleton curve, the method for determining the endpoints is as follows: traverse all pixels in the set of ordered pixels, count the number of pixels in the 8-neighborhood of each pixel that belong to the set of ordered pixels, and if the number is equal to 1, then the point is the endpoint.

[0025] In the image coordinate system, the origin is located at the top left corner, the horizontal axis points to the right as the positive direction, and the vertical axis points downwards as the positive direction. Since the base of the pineapple seedling is close to the ground and is usually located at the bottom of the image, while the tip of the seedling is located at the top of the image, the ordinate values ​​of the two endpoints are compared: the endpoint with the larger ordinate is taken as the base endpoint, and the endpoint with the smaller ordinate is taken as the top endpoint.

[0026] Starting from the base endpoint, move pixel by pixel along the skeleton curve to the top endpoint. Record all skeleton points in this order to obtain an ordered skeleton sequence. Since the skeleton curve is a connected curve with a single pixel width, the path from one end to the other is unique, so the traversal order is uniquely determined.

[0027] For each skeleton point in the ordered skeleton sequence, the tangent direction at that point needs to be calculated. This embodiment uses the local neighborhood difference method: taking two points before and after the current point as reference points, calculating the direction from the previous reference point to the next reference point, and using the direction angle of this direction as the tangent direction angle of the current point. If the index of the current skeleton point in the skeleton sequence is less than 3 or greater than n-2 (where n is the total number of skeleton points), it is impossible to take two points before and after it. In this case, the offset is adjusted to one point before and after it.

[0028] After obtaining the tangent direction, rotate this direction 90 degrees counterclockwise to obtain the first normal direction, and the opposite direction of the first normal direction to obtain the second normal direction. Since the subsequent width measurement needs to extend to both sides, the first and second normal directions are actually used for extension.

[0029] For each skeleton point in the ordered skeleton sequence, the width of the seedling region is measured by extending outwards along the normal direction in the binary mask. The specific steps are as follows: Starting from the current skeleton point, move outwards pixel by pixel along the first normal direction. After each pixel movement, check the value of that pixel in the binary mask. If the value is 1, it indicates the seedling region is still within the seedling region, so continue moving outwards; if the value is 0, it indicates the background has been reached, so stop moving and record the current distance moved. Similarly, move outwards pixel by pixel from the current skeleton point along the second normal direction until a background pixel is encountered, and record the distance moved. The width of the seedling at the current skeleton point is equal to the sum of the two moving distances, in pixels.

[0030] Arrange the seedling widths corresponding to each skeleton point in traversal order to obtain the contour width curve.

[0031] Based on an ordered skeleton sequence, the curvature value at each skeleton point is calculated, and local maxima of curvature are identified. For each skeleton point, the first and last points are excluded (i.e., from the second to the second-to-last point). The direction of the line segment from the previous skeleton point to the current point, and the direction of the line segment from the current point to the next skeleton point are calculated separately. The direction angles of the two line segments are then calculated, and the absolute value of the difference between the two direction angles is taken as the included angle value. If this difference is greater than 180 degrees, it is subtracted from 360 degrees to ensure that the included angle value is always the smaller angle between 0 and 180 degrees. This included angle is taken as the curvature value at the current point. If the curvature value of a skeleton point is greater than the curvature values ​​of both its preceding and following skeleton points, then this point is marked as a local maximum of curvature. All local maxima of curvature are collected to form a set of curvature extrema.

[0032] Step 2: Calculate the local maxima and local minima based on the contour width curve to form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as the candidate clamping point set.

[0033] Following the traversal order of the skeleton curve, the width value of the pineapple seedling body corresponding to each pixel on the contour width curve is traversed sequentially. For internal pixels on the contour width curve, excluding the first pixel (corresponding to the base endpoint) and the last pixel (corresponding to the top endpoint), local maxima and local minima of width are determined respectively. The method for determining local maxima of width is as follows: the width value of the current pixel is compared with the width values ​​of the previous and next pixels. If the width value of the current pixel is simultaneously greater than the width value of the previous pixel and simultaneously greater than the width value of the next pixel, then the current pixel is determined to be a local maximum of width.

[0034] The method for determining a local minimum in width is as follows: compare the width value of the current pixel with the width values ​​of its predecessor and successor pixels. If the width value of the current pixel is simultaneously less than the width value of its predecessor and simultaneously less than the width value of its successor pixels, then the current pixel is determined to be a local minimum in width.

[0035] All detected local maxima and local minima of width are combined to form a set of width extrema. The set of curvature extrema has already been calculated in step 1, and this set of curvature extrema can be directly used in this step.

[0036] The set of curvature extrema points and the set of width extrema points are combined to obtain the initial candidate clamping point set. Since the curvature extrema points and the width extrema points may fall on adjacent or close positions on the skeleton curve, there are multiple points in the initial candidate clamping point set that are too close in space. These points actually point to the same area on the seedling body and need to be merged and deduplicated.

[0037] The distance determination method is as follows: Calculate the straight-line distance (i.e., Euclidean distance) between any two candidate gripping points in the initial candidate gripping point set on the image plane. A distance threshold is preset. In this embodiment, the camera resolution is set to 1920×1080 pixels. Before inputting the acquired original image into the semantic segmentation network, it is first scaled to 512×512 pixels to meet the network input requirements. The average width of the pineapple seedling in the image is approximately 30 to 50 pixels. To avoid the candidate gripping points being too dense, causing the robotic arm to repeatedly grip adjacent positions, and to ensure that no truly valuable gripping points are missed, this embodiment sets the distance threshold to 10 pixels, approximately one-third to one-fifth of the average width of the seedling. It should be noted that this threshold can be adjusted according to the camera resolution, the actual size of the pineapple seedling, and the size of the gripper end. When the camera resolution decreases, the threshold can be decreased accordingly; when the seedling is wider, the threshold can be appropriately increased. If the straight-line distance between two candidate gripping points is less than this threshold, it is determined that the two points are too close and need to be merged into one point.

[0038] The merging rules are as follows: For two or more candidate gripping points that are too close together, compare the characteristic amplitudes of these points. If these points contain curvature extrema, compare the curvature values ​​of each point, retain the point with the largest curvature value, and discard the others. If these points do not contain curvature extrema but only width extrema, compare the width extrema amplitude of each point (i.e., the absolute value of the difference between the width value of this point and the width values ​​of its adjacent points), retain the point with the largest width extrema amplitude, and discard the others. If these points contain both curvature extrema and width extrema, retain the point with the larger curvature value. After merging, a set of candidate gripping points is obtained.

[0039] Table 1 shows an example of the candidate clamping point merging and deduplication process. As shown in Table 1, multiple curvature extrema and width extrema were detected on the skeleton segment. Some of these extrema are spatially close (less than the preset 10-pixel threshold) and need to be deduplicated according to the merging rules. Point 8 is a curvature extrema. Its neighboring points 10 and 12 are 6 pixels and 4 pixels away from it, respectively, both less than the threshold. The curvature value of point 8 (0.32) is higher than that of point 12 (0.18). Point 10 is a width extrema. According to the rule of "retaining the one with the larger curvature value when both curvature and width extrema are included", point 8 is ultimately retained. Points 22 and 24 are 4 pixels apart, less than the threshold. Both are curvature and width extrema, but the width extrema amplitude (10) of point 24 is larger. According to the rule of "retaining the one with the largest amplitude when only width extrema are included", point 24 is ultimately retained. Similarly, point 35 merges with nearby points 38 and 42. After merging and deduplication, the initial 9 extreme points are merged into 5 candidate gripping points, namely points 8, 18, 24, 35, and 46. This demonstrates that merging and deduplication effectively reduces redundancy among candidate gripping points, preventing the robotic arm from repeatedly gripping adjacent positions.

[0040] Table 1: Example of candidate gripping point merging and deduplication process Step 3: Construct a time window containing the current frame and several frames before the current frame. For each candidate clamping point in each frame within the time window, extract feature values ​​on multiple preset evaluation dimensions and convert the feature values ​​into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame.

[0041] A time window is defined as a set containing the current frame and several frames preceding it. The length of the time window is L frames, meaning it includes the current frame (denoted as frame t) and the L-1 preceding frames (frames t-L+1, t-L+2, ..., t-1). In this embodiment, the time window length L is 5 frames, meaning the evaluation uses the current frame and its four preceding frames. It should be noted that the value of L can be adjusted based on the camera frame rate and the robotic arm's response speed: a higher frame rate allows for a larger L value to obtain longer temporal information; a lower frame rate allows for a smaller L value to avoid excessive image changes within the window.

[0042] For each frame within the time window, the skeleton curve, contour width curve, curvature extremum point set, and width extremum point set are extracted from that frame according to steps 1 and 2, ultimately yielding the candidate clamping point set for that frame. It is worth noting that the number of candidate clamping points detected in different frames may vary, and candidate clamping points corresponding to the same physical location in different frames need to be associated through position matching. In this embodiment, if the straight-line distance between two candidate clamping points in two frames is less than 10 pixels on the image plane, then these two points are determined to be the corresponding points of the same candidate clamping point in different frames. This setting is based on the following: under the condition that the camera resolution is set to 1920×1080 pixels and the input image is scaled to 512×512 pixels, the average width of the pineapple seedling in the image is approximately 30 to 50 pixels. Due to slight shaking of the seedling or slight camera jitter between adjacent frames, the positional offset of the same physical point in the image is usually within the range of 5 to 8 pixels. Setting the threshold to 10 pixels is sufficient to cover normal inter-frame displacement while avoiding incorrectly associating different candidate points that are far apart as the same point. It should be noted that this threshold can be adjusted according to the camera resolution, image scaling ratio, distance between the camera and the seedling, and the actual swaying amplitude of the seedling: when the camera resolution is reduced or the image scaling ratio is decreased, the physical size represented by the pixel increases, and the threshold should be decreased accordingly; when the swaying amplitude of the seedling is large or the distance between the camera and the seedling is close, the threshold should be increased appropriately.

[0043] In this embodiment, candidate gripping points are evaluated from three dimensions: geometric stability, image reliability, and operational rationality.

[0044] The geometric stability dimension measures the smoothness and stability of the seedling morphology within the region where candidate clamping points are located. Regions with drastic changes in curvature or width may exhibit localized deformities, leaf occlusion, or segmentation noise, making them unsuitable as clamping points. This dimension is evaluated using two metrics: the rate of change in curvature and the rate of change in contour width. Feature extraction: For a candidate clamping point in a frame within a time window, the curvature and skeletal width values ​​at that point and its surrounding skeleton points need to be obtained first. These data were calculated in step 1 to obtain the curvature value of each point on the skeleton curve and to generate the contour width curve. Centered on the position of the current candidate clamping point on the skeleton curve, a predetermined number of adjacent skeleton points before and after it are selected to form the first local neighborhood. In this embodiment, three skeleton points before and after it are selected, plus the current point itself, for a total of seven skeleton points to form the first local neighborhood. For candidate clamping points near the base endpoint or top endpoint, if three points cannot be selected on one side, the maximum number of points actually existing are selected. The rationale for this setting is that selecting three skeleton points before and after it is possible to effectively capture the curvature and width changes of the seedling in the local area without introducing skeleton points that are too far away. If the neighborhood is too small (e.g., one point before and after), the rate of curvature change and the rate of width change are easily affected by single-point measurement noise; if the neighborhood is too large (e.g., five points before and after), it may include areas that are far away on the seedling, causing the local change trend to be smoothed out and lose sensitivity. It should be noted that the radius of this neighborhood can be adjusted according to the actual situation: when the point spacing on the skeleton curve is large, the neighborhood radius can be appropriately reduced; when the point spacing on the skeleton curve is small, the neighborhood radius can be appropriately increased. For candidate clamping points near the base endpoint or top endpoint, if three points cannot be obtained on one side, then the maximum number of points that actually exist is taken. Based on the skeleton points in the first local neighborhood, the following feature values ​​are extracted: Curvature value: The curvature value at the current candidate clamping point calculated in step 1 is directly used; Contour width value: Directly use the seedling width value at the current candidate clamping point calculated in step 1; Curvature change rate: Calculate the difference in curvature values ​​between every two adjacent skeleton points in the first local neighborhood, take the absolute value, then add all these differences together, and divide by the number of adjacent point pairs. The average value obtained is the curvature change rate. The curvature change rate reflects the degree of curvature fluctuation near the current candidate gripping point; the smaller the value, the smoother the curvature change. Width Variation Rate: Calculate the difference in seedling width between every two adjacent skeleton points within the first local neighborhood, take the absolute value, then sum all these differences and divide by the number of adjacent point pairs. The average value obtained is the width variation rate. The width variation rate reflects the degree of fluctuation in seedling width near the current candidate clamping point; the smaller the value, the smoother the width change.

[0045] Scoring rules: The geometric stability dimension score is obtained by weighted summation of the curvature change rate score and the width change rate score. A first ideal interval for the curvature change rate and a second ideal interval for the width change rate are preset.

[0046] A first ideal range for the rate of change of curvature is pre-defined, defined by a lower limit and an upper limit. In this embodiment, the first ideal range is set to 0.05 to 0.25 (in radians per pixel). This range is determined based on the following: Statistical analysis of the rate of change of curvature of a large number of pineapple seedling samples revealed that when the rate of change of curvature is below 0.05, the seedling is nearly straight in this area, lacking geometric features and easily slipping during clamping; when the rate of change of curvature is above 0.25, the seedling bends excessively, causing localized stress concentration during clamping and easily leading to damage; when the rate of change of curvature is between 0.05 and 0.25, the seedling exhibits moderate curvature, providing geometric constraints for clamping without causing damage due to excessive bending. Therefore, this range is considered the ideal range. When the curvature change rate of a candidate clamping point is within the ideal range, a preset first base score is given. In this embodiment, the first base score is 50 points. The basis for this base score is: the total score of the geometric stability dimension is 100 points (because when it is weighted and summed with image reliability and job rationality, the full score of each dimension is 100 points), of which the curvature change rate contributes 60% (weight 0.6), so the full score of the curvature change rate is 60 points. However, considering that it will be weighted with the width change rate score later, and that there is attenuation in the actual scoring process, the curvature change rate score under the ideal condition is set to 50 points, leaving a margin of 10 points to distinguish candidate points of different qualities; When the curvature change rate of a candidate clamping point is outside the ideal range, the score is linearly reduced according to the degree of deviation of the curvature change rate from the ideal range. Specifically: First, the distance from the curvature change rate to the nearest boundary of the ideal range (i.e., the lower limit of 0.05 or the upper limit of 0.25) is calculated and recorded as the deviation distance. Then, a maximum allowable deviation distance is preset, which is set to 0.30 radians per pixel in this embodiment. The basis for determining this maximum allowable deviation distance is: through extreme value analysis of sample data, the value range of the curvature change rate is usually between 0 and 0.55. 0.30 is about half of this range. When it exceeds this range, the curvature change is too extreme, and the score should be reduced to 0. Then, the deviation distance is divided by the maximum allowable deviation distance to obtain the deviation ratio. If the deviation ratio is greater than 1, it is set to 1. Finally, the curvature change rate score = first base score × (1 - deviation ratio); It should be noted that the upper and lower limits of the first ideal interval can be adjusted according to the pineapple seedling variety: different varieties of pineapple seedlings have different bending characteristics. If the rate of change of curvature of some upright varieties is generally small, the ideal interval should be appropriately shifted downward; if the rate of change of curvature of some creeping varieties is generally large, the ideal interval should be appropriately shifted upward. The maximum allowable deviation distance can also be adjusted according to the actual sample distribution.

[0047] Width Change Rate Scoring Rules: A second ideal range for the width change rate is pre-defined, defined by a lower limit and an upper limit. In this embodiment, the second ideal range is set to 0.02 to 0.10 (in pixels per pixel). This range is determined based on the following: Statistical analysis of the width change rate of a large number of pineapple seedling samples revealed that when the width change rate is below 0.02, the seedling width remains almost constant in this area, lacking geometric constraints in the width direction; when the width change rate is above 0.10, the seedling width changes too drastically (e.g., leaves suddenly appear or the seedling suddenly narrows), and this area may have leaf occlusion or seedling deformities, making it unsuitable for clamping; when the width change rate is between 0.02 and 0.10, the seedling width changes gradually, resulting in uniform force during clamping. Therefore, this range is considered the ideal range. When the width change rate of the candidate clamping point is within the ideal range, a preset second basic score is given. In this embodiment, the second basic score is 50 points, and the basis for the value is similar to that of the curvature change rate: the weight of the width change rate in the geometric stability dimension is 0.4, the full score is 40 points in the ideal case, and 50 points is set after leaving a margin; When the width change rate of a candidate clamping point is outside the ideal range, the score is linearly reduced according to the degree to which the width change rate deviates from the ideal range. The maximum allowable deviation distance for the preset width change rate is 0.15 pixels per pixel. This value is determined based on the fact that the width change rate typically ranges from 0 to 0.25, and 0.15 is approximately two-thirds of this range. Beyond this range, the width change is considered too extreme. The deviation ratio and score are calculated in the same way as the curvature change rate. It should be noted that the upper and lower limits of the second ideal range can be adjusted according to the pineapple seedling variety and growth stage: the width of pineapple seedlings changes rapidly during the seedling stage, so the ideal range should be appropriately widened; the width of pineapple seedlings tends to stabilize during the mature stage, so the ideal range can be appropriately narrowed.

[0048] The specific logic of weighted summation is as follows: The curvature change rate score and the width change rate score are weighted and summed according to preset weights to obtain the geometric stability dimension score. In this embodiment, the weight of the curvature change rate score is set to 0.6, and the weight of the width change rate score is set to 0.4. The basis for determining this weight is that in the geometric stability assessment, curvature change reflects the bending shape of the seedling and contributes more to the geometric constraint of the clamping point; width change reflects the thickness change of the seedling and contributes relatively less. Therefore, the curvature change rate is given a higher weight. It should be noted that this weight can be adjusted according to the actual application scenario: if the clamp is more sensitive to the width change of the seedling, the weight of the width change rate should be appropriately increased; if the clamp is more sensitive to the bending shape, the weight of the curvature change rate should be maintained or increased.

[0049] The image reliability dimension measures the image quality of the local region where the candidate gripping point is located. Regions with excessively high edge density usually correspond to overlapping leaves, shadow boundaries, or image noise; the segmentation results in these regions are unreliable and should be avoided.

[0050] The specific logic for feature extraction is as follows: Centered on the candidate clamping point, a local image patch of a preset size is extracted, which is essentially a square region cropped from the original image. In this embodiment, the size of the local image patch is set to 31×31 pixels.

[0051] Edge detection is performed on the local image patch. The purpose of edge detection is to identify locations in the image where grayscale values ​​change drastically; these locations typically correspond to seedling boundaries, leaf edges, or shadows caused by light. This embodiment uses the Canny edge detection algorithm, which includes the following steps: first, the image is smoothed and denoised using a Gaussian filter; then, the gradient magnitude and direction of each pixel are calculated; next, non-maximum suppression is applied to the gradient magnitude, retaining only the points with the largest local gradients; finally, a double thresholding method is used to connect the edges, resulting in a binary edge image. In the binary edge image, edge pixels have a value of 1, and non-edge pixels have a value of 0.

[0052] The total number of edge pixels within a local image patch is counted, which is the number of pixels with a value of 1 in the binary edge image. The ratio of this number of edge pixels to the total number of pixels in the local image patch is calculated, which in this embodiment is 31 × 31 = 961 pixels. This ratio is the image edge density. The specific logic of the scoring rules is as follows: An upper limit threshold for edge density is preset. In this embodiment, the upper limit threshold for edge density is set to 0.15. The determination of this threshold is based on the following: Through statistical analysis of the edge density of reliable regions (i.e., regions manually marked as suitable for clamping) and unreliable regions (e.g., overlapping leaves, shadow boundaries, and noise areas) in a large number of pineapple seedling images, it was found that the edge density of reliable regions is usually between 0.05 and 0.12, while the edge density of unreliable regions is usually above 0.15. Setting the threshold to 0.15 can effectively distinguish between reliable and unreliable regions. At the same time, this threshold is slightly higher than the upper limit of reliable regions to avoid misjudging reliable regions as unreliable due to slight edge fluctuations.

[0053] When the edge density of a candidate clamping point is less than the upper threshold of edge density, the image quality of that local region is considered reliable. The image reliability dimension score is obtained by linearly mapping the difference between the edge density and the upper threshold. Specifically: Image reliability dimension score = 100 × (1 - edge density / upper threshold of edge density). The reason for using a linear mapping is that image quality generally decreases linearly from 0 to the upper threshold: the lower the edge density, the cleaner the image, and the higher the score should be; the closer the edge density is to the upper threshold, the cluttered the image, and the lower the score should be.

[0054] When the edge density of a candidate clamping point is greater than or equal to the upper limit threshold of edge density, the image quality of that local region is considered unreliable, and the image reliability dimension score is directly set to 0. This is because once the edge density exceeds the threshold, the region is highly likely to belong to leaf overlap or shadow boundary, and should not be used as a clamping point regardless of the score.

[0055] It should be noted that the upper limit threshold for edge density can be adjusted according to the image acquisition conditions: in a greenhouse environment with uniform lighting and a clean background, the threshold can be appropriately lowered to select higher quality areas; in a field environment with complex lighting and a cluttered background, the threshold can be appropriately increased to avoid all candidate points scoring too low due to environmental factors.

[0056] The operational rationality dimension measures whether the position of the candidate gripping point on the seedling is suitable for the robotic arm to perform gripping operations. Too close a distance from the base may damage the seedling's growth point, while too far a distance results in a seedling that is too thin or too soft, leading to poor gripping stability. This dimension is evaluated by the ratio of the arc length from the candidate gripping point to the base endpoint to the total arc length of the skeleton.

[0057] The specific logic for feature extraction is as follows: The skeleton curve has been obtained in the above steps, and the base endpoint and top endpoint have been determined. First, calculate the arc length distance from the candidate clamping point to the base endpoint along the skeleton curve. The arc length distance is the Euclidean distance between adjacent points along the skeleton curve, accumulated point by point. Starting from the base endpoint, the distance from each skeleton point to the next skeleton point is accumulated until the candidate clamping point is reached. The accumulated result is the arc length distance.

[0058] Then calculate the total arc length of the skeleton curve, which is the sum of the Euclidean distances between all adjacent points along the skeleton curve from the base endpoint to the top endpoint.

[0059] The distance ratio is calculated by dividing the arc length from the candidate clamping point to the base endpoint by the total arc length of the skeleton curve. The distance ratio ranges from 0 to 1, where 0 indicates that the candidate clamping point is the base endpoint itself, and 1 indicates that the candidate clamping point is the top endpoint itself.

[0060] The specific logic of the scoring rules is as follows: Pre-set an optimal distance ratio and a Gaussian function width parameter.

[0061] The optimal distance ratio refers to the proportion of the plant from the base upwards that is most suitable for clamping. In this embodiment, the optimal distance ratio is set to 0.35, which is approximately one-third of the way up from the base. This value is determined based on the following: through research on the agronomic characteristics of pineapple seedlings and clamping experiments, it was found that if the distance is too close to the base (i.e., the distance ratio is less than 0.2), the base of the seedling is relatively thick and close to the growing point, which may damage the seedling during clamping; if the distance is too far from the base (i.e., the distance ratio is greater than 0.5), there are more leaves on the upper part of the seedling and the seedling becomes thinner, resulting in poor clamping stability; when the distance ratio is between 0.30 and 0.40, the seedling is of moderate thickness, has relatively few leaves, and is far from the growing point, which is the optimal clamping area. In this embodiment, the median value of 0.35 is taken as the optimal value.

[0062] The Gaussian function width parameter controls the rate at which the score decays as the distance percentage deviates from the optimal value. In this embodiment, the width parameter is set to 0.12. This value is determined as follows: when the distance percentage deviates from the optimal value by 0.12, the score decays to approximately 60% of its maximum value; when the distance percentage deviates from the optimal value by 0.24, the score decays to approximately 13% of its maximum value. This decay rate ensures that the score remains at a relatively high level within a 0.12 range near the optimal value, and decreases rapidly beyond this range, consistent with the actual principle that "the optimal clamping area is narrow, and the clamping effect decreases significantly after deviation."

[0063] The assignment rationality score is calculated using a Gaussian function. The input to this Gaussian function is the distance percentage of the candidate clamping points, and the output is the assignment rationality score. The Gaussian function is centered on the optimal distance percentage, which refers to the proportion of the distance from the base upwards that is most suitable for clamping. This value is determined based on the agronomic characteristics of the pineapple seedlings and clamping experiments. The Gaussian function also includes a width parameter to control the rate at which the score decays as the distance percentage deviates from the optimal value.

[0064] It should be noted that both the optimal distance ratio and the Gaussian function width parameter can be adjusted according to the pineapple seedling variety, growth stage, and clamp type. For example, for young pineapple seedlings, the optimal clamping position should be closer to the base, such as a distance ratio of 0.25, to avoid damaging the tender upper tissues; for taller varieties, the optimal clamping position can be moved slightly upwards, such as a distance ratio of 0.40; for clamps with wider openings, the width parameter can be increased appropriately to expand the acceptable range; for scenarios requiring high clamping precision, the width parameter can be decreased appropriately to strictly limit the clamping position.

[0065] For each candidate gripping point in each frame of an image within a time window, after calculating the geometric stability dimension score, image reliability dimension score, and operational rationality dimension score respectively, the scores are weighted and summed according to a preset comprehensive weight to obtain the comprehensive score of the candidate gripping point in that frame of the image.

[0066] In this embodiment, the weights of the three dimensions are set as follows: geometric stability dimension has a weight of 0.4, image reliability dimension has a weight of 0.3, and operational rationality dimension has a weight of 0.3. The basis for these weights is as follows: geometric stability directly reflects the morphological characteristics of the seedling itself and is the core basis for selecting the gripping point, therefore it is given the highest weight of 0.4; image reliability reflects image quality and has a significant impact on the accuracy of the segmentation results, but it can be compensated to some extent through image preprocessing and temporal information, therefore it is given a medium weight of 0.3; operational rationality reflects whether the gripping position is suitable for the robotic arm's operation. Although important, it can be adjusted according to the robotic arm parameters, therefore it is also given a weight of 0.3. It should be noted that these weights can be adjusted according to the actual application scenario. If the camera's imaging quality is poor, the weight of the image reliability dimension can be appropriately increased; if the structure of the robotic arm gripper is special, the weight of the operational rationality dimension can be appropriately increased; if the pineapple seedling's morphology varies greatly, the weight of the geometric stability dimension can be appropriately increased.

[0067] The formula for calculating the overall score is: Overall Score = 0.4 × Geometric Stability Dimension Score + 0.3 × Image Reliability Dimension Score + 0.3 × Job Reasonableness Dimension Score. The overall score ranges from 0 to 100 points, with a higher score indicating better overall quality of the candidate gripping point in that frame of image.

[0068] Table 2 shows the evolution of the comprehensive scores of each candidate clamping point within the time window. According to Table 2, this embodiment uses the five candidate clamping points (corresponding to skeleton points 8, 18, 24, 35, and 46) obtained after merging and deduplication in Table 1 as examples to demonstrate their comprehensive score evolution within a five-frame time window. As shown in Table 2, candidate point A (point 8) consistently maintains a high comprehensive score within the time window, but exhibits a slow downward trend, decreasing from 82.5 points in frame t-4 to 78.5 points in frame t; candidate point B (point 18) has a low comprehensive score but shows a continuous upward trend, rising from 65.0 points to 72.5 points; candidate points C (point 24) and D (point 35) have moderate comprehensive scores, with C slightly decreasing and D steadily increasing; candidate point E (point 46) has the lowest comprehensive score and continues to decrease, dropping from 60.0 points to 48.0 points. The multi-frame score data within the time window shows that relying solely on a single frame score (such as frame t) may misjudge the true quality of the candidate points. For example, candidate point A has the highest score (78.5 points) in frame t, but its score shows a downward trend; candidate point B has a lower score than A, but its score shows an upward trend. Therefore, a comprehensive evaluation combining temporal stability score and future sustainability score is needed.

[0069] According to Table 2 and Figure 2 It can be seen that the overall score change trends of different candidate clamping points within the time window are significantly different. The score of candidate point C (corresponding to skeleton point 24) remained basically stable (from 74.5 to 74.5), with a change of almost 0 within 5 frames, indicating that this candidate point has the best score consistency across different frames and is least affected by image noise, lighting changes, or seedling swaying. The score of candidate point D (corresponding to skeleton point 35) increased from 74.0 to 74.5, showing a slight upward trend, indicating that the quality of this candidate point is slowly improving. The scores of candidate points A, B, and E all showed a downward trend, with a decrease of about 4.0 points within 5 frames. Among them, candidate point A had the highest initial score (88.0 points), but after the decrease, it still remained at 84.0 points, which is still at a relatively high level. Figure 2 It can be intuitively seen that even if all candidate points have scores of the same order of magnitude, the differences in their trends (stable, rising, falling) and the rate of decline provide important basis for the subsequent calculation of time series stability scores and future sustainability scores. For example, the stable trend of candidate point C indicates that its future sustainability is relatively good, while the declining trend of candidate point A needs to be discounted by the future sustainability score.

[0070] Table 2: Evolution of the overall score of each candidate clamping point within the time window Table 3 shows the three-dimensional and comprehensive scores of each candidate clamping point in frame t. According to Table 3, candidate point A has high scores in geometric stability (80 points) and image reliability (85 points), but its operational rationality score (70 points) is relatively low. This is because the point is close to the base (distance ratio is approximately 0.25, deviating from the optimal value by 0.35), resulting in a deduction in the operational rationality dimension. Candidate point B has the highest operational rationality score (80 points), but a low geometric stability score (65 points). This is because the curvature change rate near this point is large (approximately 0.28 radians per pixel, exceeding the upper limit of the ideal range by 0.25). Candidate point D has relatively balanced scores across the three dimensions (geometric stability 70 points, image reliability 80 points, operational rationality 75 points), with a comprehensive score of 74.5 points. Table 3 shows that the three dimensions evaluate the candidate clamping points from different perspectives, complementing each other and avoiding the bias that might arise from a single indicator.

[0071] Table 3: Three-dimensional scores and comprehensive scores of each candidate clamping point in frame t Step 4: Based on the comprehensive scores of each candidate clamping point in each frame image within the time window, construct a comprehensive score sequence corresponding to each candidate clamping point. Based on the comprehensive score sequence, calculate the temporal stability score of each candidate clamping point, and calculate the future sustainability score of each candidate clamping point according to the changing trend of the comprehensive score sequence.

[0072] For each candidate gripping point, a comprehensive score is extracted from each frame within the time window. Since candidate gripping points in different frames need to be associated through position matching, it can be ensured that the scores of the same candidate gripping point can be correctly matched in different frames.

[0073] The extracted comprehensive scores are arranged in chronological order according to the frames to form a comprehensive score sequence for each candidate clamping point. Let the time window length be L frames; then the comprehensive score sequence contains L score values, corresponding sequentially to frames t-L+1, t-L+2, ..., t (where frame t is the current frame). In this embodiment, L is set to 5, therefore the comprehensive score sequence for each candidate clamping point contains 5 score values.

[0074] Temporal stability score is used to measure the degree of fluctuation in the overall score of candidate clamping points within a time window. The smaller the score fluctuation, the more consistent the performance of the candidate clamping point is in different frame images, and the less it is affected by factors such as noise, illumination changes, and seedling shaking, thus making it more suitable as a clamping point.

[0075] This step uses variance as an indicator of the degree of fluctuation. The variance is calculated as follows: First, calculate the average of all scores in the comprehensive score sequence; then, calculate the square of the difference between each score and the average; finally, sum these squared values ​​and divide by the number of scores. The result is the variance. A larger variance indicates more volatile scores; a smaller variance indicates more stable scores.

[0076] After calculating the variance, it is matched with multiple preset variance intervals, each interval corresponding to a time series stability score. In this embodiment, the preset variance intervals and their corresponding time series stability scores are as follows: Variance Interval 1: Variance greater than or equal to 0 and less than or equal to 50. The temporal stability score corresponding to this interval is 1.0. The basis for setting this interval is as follows: Through statistical analysis of a large number of candidate gripping points with stable tracking, it was found that when the variance of the comprehensive score is within 50, the score fluctuation is small, and the quality of the candidate gripping points remains highly consistent across different frames. Therefore, the highest temporal stability score of 1.0 is given. Variance Interval 2: Variance greater than 50 and less than or equal to 150. The temporal stability score corresponding to this interval is 0.7. The basis for setting this interval is as follows: When the variance is between 50 and 150, the score fluctuates to some extent, but it is still within an acceptable range. The candidate gripping points may be affected by slight image noise or seedling shaking, but they are still reliable, so a score of 0.7 is given. Variance Interval 3: Variance greater than 150 and less than or equal to 300. The time series stability score corresponding to this interval is 0.4. The basis for setting this interval is as follows: When the variance is between 150 and 300, the score fluctuates more significantly, the candidate pinch points may be subject to strong interference, and the reliability is low, so a score of 0.4 is given; Variance Interval 4: Variance greater than 300. The temporal stability score corresponding to this interval is 0.1. The basis for setting this interval is as follows: Through the analysis of a large number of unstable candidate gripping points, a variance exceeding 300 means that the score fluctuates drastically between different frames, and the candidate gripping point is very likely located at the edge of the blade, the shadow area, or other unreliable location, so an extremely low score of 0.1 is given.

[0077] It should be noted that the boundary values ​​(50, 150, 300) of the above variance intervals were determined based on the characteristic that the comprehensive score range in this embodiment is from 0 to 100 points, combined with the statistical analysis results of a large amount of experimental data. When the range of the comprehensive score changes, these boundary values ​​should be adjusted proportionally accordingly. For example, if the comprehensive score range is from 0 to 200 points, the boundary values ​​of the variance intervals should be multiplied by 2; if a scoring scale of 0 to 50 points is used, the boundary values ​​should be multiplied by 0.5.

[0078] The future sustainability score measures the potential trend of the candidate gripper's overall score over a future period. An upward trend in the overall score indicates that the candidate gripper's quality is improving and its future sustainability is good; a downward trend indicates that its quality is deteriorating and its future sustainability is poor; and a stable score indicates that its quality is stable.

[0079] This step uses the least squares method to fit a linear trend line to assess the changing trend of the scores. The specific method is as follows: Using the frame number within the time window as the independent variable and the overall score corresponding to each frame as the dependent variable, a straight line is fitted to minimize the sum of squares of the differences between the actual score of each frame and the score at the corresponding point on the line. The slope of this line is the trend slope: a positive slope indicates that the score increases over time, a negative slope indicates that the score decreases over time, and a larger absolute value of the slope indicates a faster change.

[0080] After calculating the trend slope, the slope is matched with multiple preset slope intervals, each interval corresponding to a future sustainability score. In this embodiment, the preset slope intervals and their corresponding future sustainability scores are as follows: Slope Interval 1: Slope greater than or equal to 0.05. The future sustainability score corresponding to this interval is 1.2. The basis for setting this interval is as follows: When the slope is greater than or equal to 0.05, it indicates that the comprehensive score shows a significant upward trend within the time window, the quality of the candidate clamping points is continuously improving, and the future sustainability is very good. Therefore, a gain score of 1.2 greater than 1 is given. Slope Interval Two: Slope greater than or equal to 0 and less than 0.05. The future sustainability score corresponding to this interval is 1.0. The basis for setting this interval is as follows: When the slope is positive but less than 0.05, it indicates that the overall score has a slight upward trend or remains basically stable, the quality of the candidate clamping points is stable or slowly improving, and the future sustainability is good, so a score of 1.0 is given; Slope Interval 3: Slope less than 0 and greater than or equal to -0.05. The future sustainability score corresponding to this interval is 0.8. The basis for setting this interval is as follows: When the slope is negative but the absolute value is less than or equal to 0.05, it indicates that the overall score has a slight downward trend, the quality of the candidate gripping points is slowly deteriorating, and the future sustainability is generally average, so a score of 0.8 is given; Slope Interval 4: Slope less than -0.05. The corresponding future sustainability score for this interval is 0.5. The basis for setting this interval is as follows: When the slope is less than -0.05, it indicates that the overall score shows a significant downward trend, the quality of the candidate gripping points is rapidly deteriorating, and the future sustainability is poor, therefore a low score of 0.5 is given.

[0081] The determination of the slope interval boundary value, 0.05, is based on the following: In this embodiment, the comprehensive score ranges from 0 to 100 points, the time window length is 5 frames, and the maximum possible score change between two adjacent frames is usually within 10 points. A slope of 0.05 corresponds to a total change of 0.25 points across 5 frames, which is a very small change used to distinguish between "stable" and "significant change." It should be noted that this boundary value can be adjusted according to the time window length and the actual change in the comprehensive score: the longer the window length, the smaller the absolute value of the slope corresponding to the same trend of change, so the boundary value should be appropriately reduced; conversely, the shorter the window length, the more the boundary value should be appropriately increased.

[0082] Temporal stability score and future sustainability score will be used in the weighted fusion calculation of subsequent steps. The temporal stability score, as a discount factor, ranges from 0 to 1 and is used to reduce the comprehensive score of the current frame: the worse the stability, the greater the discount, and the lower the final score. The future sustainability score, as a gain factor, can range from greater than 1 to less than 1 and is used to gain the discounted score: the gain is greater than 1 when the trend is positive, and less than 1 when the trend is negative.

[0083] It should be noted that the number of divisions, boundary values, and corresponding scores of the above variance and slope intervals can be adjusted according to the actual application scenario. If high stability is required, the variance interval corresponding to high stability can be narrowed, and the score corresponding to low stability can be further reduced. If the confidence of predicting future trends is low, the range of future sustainability scores can be narrowed to avoid the gain factor having too much influence on the final result.

[0084] Table 4 shows the comprehensive score sequence of each candidate point. According to Table 4, the variance of all candidate points is between 2.2 and 16.4, all less than 50. Therefore, the time series stability score is 1.0 for all five candidate points, indicating that the comprehensive score fluctuations within the time window are small and the historical stability is good. However, there are significant differences in the future sustainability scores of the candidate points: the slopes of candidate points B and D are +1.85 and +1.15, respectively, both greater than 0.05, with a future sustainability score of 1.2, indicating a significant upward trend in their comprehensive scores and continued improvement in future quality; the slopes of candidate points A and E are -1.05 and -3.00, respectively, both less than -0.05, with a future sustainability score of 0.5, indicating a significant downward trend in their comprehensive scores and a potential continued deterioration in future quality; the slope of candidate point C is -0.75, between -0.05 and 0, with a future sustainability score of 0.8, indicating a slight downward trend in its comprehensive score. It is evident that the temporal stability score alone cannot distinguish these five candidate points (all 1.0), but the future sustainability score can effectively reveal the differences in the development trends of each candidate point, providing key information for subsequent weighted fusion.

[0085] Table 4: Comprehensive score sequence of each candidate point Step 5: Perform a weighted fusion of the comprehensive score, temporal stability score, and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point.

[0086] This step employs a fusion strategy combining discount factors and gain factors. Specifically: the comprehensive score of the candidate clamping point in the current frame, i.e., the last frame of the time window, is used as the base value; the temporal stability score is used to reduce this base value, because candidate points with poor stability, even if they have a high score in the current frame, may be due to random noise and should not be given too much trust; then the future sustainability score is used to gain the discounted result, i.e., amplify or reduce it, because candidate points with a positive trend will have higher future quality, while candidate points with a negative trend will have lower future quality.

[0087] The advantage of this fusion strategy lies in the fact that the three scores evaluate candidate clamping points from different time dimensions: the current frame score reflects immediate quality, the temporal stability score reflects historical consistency, and the future sustainability score reflects the trend of change. These three scores are independent yet complementary, effectively avoiding the bias caused by a single-dimensional evaluation.

[0088] The temporal stability score for each candidate clamping point has been calculated in the steps above. This score ranges from 0 to 1; a higher score indicates greater stability and less fluctuation in the overall score of the candidate clamping point within the time window, while a lower score indicates greater fluctuation and lower reliability. Since this score is already within the range of 0 to 1, it can be directly used as a discount factor without additional normalization.

[0089] The discount factor reduces the overall score of the current frame. When the discount factor is close to 1, the overall score of the current frame is fully preserved or only slightly reduced; when the discount factor is close to 0, the overall score of the current frame is significantly reduced. Even if the candidate point has a high score in the current frame, its first intermediate score will be lowered, thus reducing the probability of the candidate point being selected.

[0090] It should be noted that if the range of values ​​for the time series stability score is adjusted in practical applications, normalization is required to map all scores to the range of 0 to 1. The normalization method is as follows: divide the time series stability score of each candidate point by the maximum value of the time series stability scores among all candidate points, so that the normalized value falls between 0 and 1.

[0091] The first intermediate score is obtained by multiplying the overall score of the candidate gripping point in the last frame of the time window by a discount factor. The calculation formula is: First intermediate score = Overall score of the last frame × Temporal stability score.

[0092] This calculation means that the first intermediate score of a candidate pinch point depends on two factors: its overall quality in the current frame and its historical stability. The first intermediate score will be high only when both are high; if one factor is low, the first intermediate score will be lowered. The purpose of this design is to effectively suppress candidate points that perform well in the current frame but have large historical fluctuations.

[0093] Since the sustainability scores of each candidate clamping point obtained in step 4 are already within a directly usable range centered at 1.0, they are directly used as gain factors. The role of the gain factor is to amplify or reduce the first intermediate score. When the gain factor is greater than 1, the first intermediate score is amplified, indicating that the quality of the candidate point is improving and deserves higher trust; when the gain factor is equal to 1, the first intermediate score remains unchanged, indicating that the quality of the candidate point is stable; when the gain factor is less than 1, the first intermediate score is reduced, indicating that the quality of the candidate point is declining and trust should be reduced.

[0094] It should be noted that if the range of values ​​for the future sustainability score is adjusted in practical applications, normalization is required to map all scores to a range centered at 1.0. The normalization method is as follows: first, divide all scores by a preset baseline value, and then perform a shift adjustment as needed so that the scores corresponding to a stable trend are mapped to 1.0.

[0095] The final comprehensive score of the candidate clamping point is obtained by multiplying the first intermediate score by the gain factor. The calculation formula is: Final comprehensive score = First intermediate score × Future sustainability score = Last frame comprehensive score × Temporal stability score × Future sustainability score.

[0096] This calculation means that the final score of a candidate gripping point depends on the combined effect of three factors: the overall quality of the current frame (overall score of the last frame), historical stability (temporal stability score), and future trend (future sustainability score). These three factors are independent of each other; poor performance in any one factor will lower the final score. The purpose of this design is to enable the system to comprehensively weigh information from these three time dimensions and select the truly optimal gripping point.

[0097] After calculating the final comprehensive score for each candidate gripping point, these scores are sorted from high to low. The candidate gripping point with the highest final comprehensive score is selected as the optimal candidate gripping point. The coordinates of the optimal candidate gripping point in the image are then converted into the target position coordinates of the robotic arm end effector.

[0098] Table 5 shows the final comprehensive score calculation and optimal selection for each candidate point. According to Table 5, although candidate point A has the highest comprehensive score in frame t (78.5 points), its future sustainability score is only 0.5 (showing a downward trend), resulting in a significant drop in the final comprehensive score to 39.25 points. Candidate point D has a comprehensive score of 74.5 points in frame t, lower than candidate point A, but its future sustainability score is 1.2 (showing an upward trend), resulting in a final comprehensive score of 89.40 points, surpassing it to become the first place. Candidate point B has a final comprehensive score of 87.00 points, ranking second. According to the selection rules of this method, candidate point D with the highest final comprehensive score is determined as the optimal candidate clamping point. This result fully demonstrates the core advantages of this invention: it not only focuses on the immediate quality of the current frame but also corrects candidate points through temporal stability scores and future sustainability scores, avoiding the selection of points that, although currently performing well, are deteriorating, and prioritizing points with stable or continuously improving quality.

[0099] Table 5: Calculation of Final Comprehensive Score and Optimal Selection for Each Candidate Point Please see Figure 3The present invention also provides a vision-guided precision operation device for a pineapple seedling clamping robotic arm. The device is used to execute the aforementioned vision-guided precision operation method for the pineapple seedling clamping robotic arm, comprising: The feature extraction module is used to acquire pineapple seedling images, perform target segmentation on the pineapple seedling images to obtain pineapple seedling regions, extract skeleton curves and contour width curves based on the pineapple seedling regions, and calculate local maxima of curvature based on the skeleton curves to form a set of curvature extrema points. The candidate point generation module is used to calculate local maxima and local minima based on the contour width curve, form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as a set of candidate clamping points. The scoring calculation module is used to construct a time window containing the current frame and several frames of images before the current frame. For each candidate clamping point in each frame of the time window, feature values ​​on multiple preset evaluation dimensions are extracted, and the feature values ​​are converted into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame of the image. The temporal evaluation module is used to construct a comprehensive score sequence corresponding to each candidate clamping point based on the comprehensive score of each candidate clamping point in each frame image within the time window, calculate the temporal stability score of each candidate clamping point based on the comprehensive score sequence, and calculate the future sustainability score of each candidate clamping point based on the changing trend of the comprehensive score sequence. The weighted fusion module is used to perform weighted fusion of the comprehensive score, temporal stability score and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and to determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point.

[0100] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0101] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0103] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A vision-guided precision operation method for a pineapple seedling gripping robotic arm, characterized in that, The specific steps include: Step 1: Obtain the pineapple seedling image, perform target segmentation on the pineapple seedling image to obtain the pineapple seedling region, and extract the skeleton curve and contour width curve based on the pineapple seedling region. Calculate the local maxima of curvature based on the skeleton curve to form a set of curvature extrema points. Step 2: Calculate the local maxima and local minima based on the contour width curve to form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as the candidate clamping point set; Step 3: Construct a time window containing the current frame and several frames before the current frame. For each candidate clamping point in each frame within the time window, extract feature values ​​on multiple preset evaluation dimensions and convert the feature values ​​into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame. Step 4: Based on the comprehensive scores of each candidate clamping point in each frame image within the time window, construct a comprehensive score sequence corresponding to each candidate clamping point. Based on the comprehensive score sequence, calculate the temporal stability score of each candidate clamping point, and calculate the future sustainability score of each candidate clamping point according to the changing trend of the comprehensive score sequence. Step 5: Perform a weighted fusion of the comprehensive score, temporal stability score, and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point; The preset evaluation dimensions include: geometric stability dimension, image reliability dimension, and task rationality dimension; In the geometric stability dimension, the geometric stability dimension is determined by the curvature change rate and the contour width change rate at the candidate clamping point. A first local neighborhood is formed by a preset number of adjacent pixels before and after the candidate clamping point on the skeleton curve. The average value of the curvature value difference between adjacent pixels in the first local neighborhood is calculated as the curvature change rate, and the average value of the body width difference between adjacent pixels in the first local neighborhood is calculated as the contour width change rate. In the image reliability dimension, the image reliability dimension is determined by the image edge density of the local area where the candidate clamping point is located. A second local neighborhood of a preset size is constructed with the candidate clamping point as the center. Edge detection is performed on the second local neighborhood, and the ratio of the number of edge pixels to the total number of pixels in the second local neighborhood is used as the image edge density. In the dimension of operational rationality, the dimension of operational rationality is determined by the proportion of the arc length distance from the candidate clamping point along the skeleton curve to the base endpoint of the pineapple seedling to the total arc length of the pineapple seedling skeleton curve. The arc length distance from the candidate clamping point along the skeleton curve to the base endpoint of the pineapple seedling is calculated, and the ratio of the arc length distance to the total arc length of the pineapple seedling skeleton curve is used as the distance proportion. For the geometric stability dimension, the curvature value and contour width value at the candidate clamping point are extracted, and the curvature difference and width difference between the candidate clamping point and the previous and next skeleton points on the skeleton curve are calculated respectively. The average value of the curvature difference is used as the curvature change rate, and the average value of the width difference is used as the width change rate. The curvature value, contour width value, curvature change rate, and width change rate are used as the feature values ​​of the geometric stability dimension. For the image reliability dimension, a local image block of a preset size is extracted with the candidate clamping point as the center. Edge detection is performed on the local image block, the number of edge pixels detected is counted, and the ratio between the number of edge pixels and the total number of pixels in the local image block is calculated. The ratio is used as the feature value of the image reliability dimension. For the dimension of operational rationality, the arc length distance from the candidate clamping point to the base endpoint is calculated along the skeleton curve, and the total arc length of the skeleton curve is calculated. The ratio of the arc length distance to the total arc length is used as the feature value of the operational rationality dimension. For the geometric stability dimension, a first ideal range for the rate of curvature change and a second ideal range for the rate of width change are preset respectively. When the rate of curvature change of the candidate clamping point is within the first ideal range, the preset first basic score is used as the rate of curvature change score. When the rate of curvature change of the candidate clamping point is outside the first ideal interval, the rate of curvature change score is reduced according to the linear decay rule based on the distance between the rate of curvature change of the candidate clamping point and the boundary of the first ideal interval. When the width change rate of the candidate clamping point is within the second ideal range, the preset second basic score will be used as the width change rate score. When the width change rate of the candidate clamping point is outside the second ideal interval, the width change rate score is reduced according to the linear decay rule based on the distance between the width change rate of the candidate clamping point and the boundary of the second ideal interval. The curvature change rate score and width change rate score are weighted and summed according to the preset first and second weights to obtain the geometric stability dimension score.

2. The visual-guided precision operation method for the pineapple seedling clamping robotic arm according to claim 1, characterized in that: The skeleton curve and contour width curve of the pineapple seedling are extracted from the pineapple seedling area. The specific steps are as follows: Morphological skeletonization was performed on the binary mask image of the pineapple seedling area. An iterative erosion algorithm was used until a connected curve with a single pixel width was obtained, which was then used as the skeleton curve of the pineapple seedling. Identify the two endpoints of the skeleton curve. Based on the position of the endpoints in the binary mask, the endpoint with the larger ordinate is taken as the base endpoint, and the endpoint with the smaller ordinate is taken as the top endpoint. The order of the skeleton curve from the base endpoint to the top endpoint is used as the traversal order of the skeleton curve. Each curve position is traversed in turn, the tangent direction at each curve position is calculated, and the normal direction perpendicular to the tangent direction is determined. Extending along the normal direction to both sides in the binary mask image, measure the pixel distance between one side boundary and the other side boundary of the pineapple seedling area at the corresponding curve position, and use it as the width of the seedling at that curve position; The seedling widths at each curve position are arranged according to the traversal order along the skeleton curve to obtain the outline width curve.

3. The visual-guided precision operation method for the pineapple seedling clamping robotic arm according to claim 2, characterized in that: The extreme points of curvature on the skeleton curve and the extreme points of width on the contour width curve are calculated as candidate clamping points. The specific steps are as follows: Following the traversal order of the skeleton curve, each pixel within the skeleton curve is traversed sequentially. Based on the current pixel and its adjacent pixels, a local tangent direction is fitted, and the curvature value at the current pixel is calculated according to the change in direction of the adjacent local tangent directions. Pixels with curvature values ​​greater than the curvature values ​​of the adjacent pixels on both sides are selected as local curvature maxima. The set of curvature extrema points is formed by the local curvature maxima points. Iterate through each pixel point along the width curve of the contour and obtain the width value at each pixel point. Select the pixel point whose width value is greater than the width values ​​of the adjacent pixels on both sides as the local maximum value point of the width, and select the pixel point whose width value is less than the width values ​​of the adjacent pixels on both sides as the local minimum value point of the width. Merge the local maximum value point of the width and the local minimum value point of the width to form a set of width extreme value points. The set of curvature extrema points and the set of width extrema points are combined into a union. For adjacent points in the union whose Euclidean distance is less than a preset distance threshold, points with larger corresponding curvature values ​​or width extrema amplitudes are merged and deduplicated to obtain a set of candidate clamping points.

4. The visual-guided precision operation method for the pineapple seedling clamping robotic arm according to claim 1, characterized in that: According to the scoring rules, the feature values ​​of each dimension are converted into corresponding dimension scores, and the image reliability dimension and job rationality dimension scores of each candidate gripping point in this frame are calculated. The specific steps are as follows: For the image reliability dimension, a preset upper limit threshold for edge density is set. When the edge density of the candidate clamping point is less than the upper limit threshold, a linear mapping is performed according to the difference between the edge density and the upper limit threshold to obtain the image reliability dimension score. When the edge density of the candidate clamping point is greater than or equal to the upper limit threshold, the image reliability dimension score is set to zero. For the task rationality dimension, the optimal value of distance ratio and the width parameter of Gaussian function are preset, and the distance ratio of candidate clamping points is input into the preset Gaussian function to obtain the task rationality dimension score; The geometric stability dimension score, image reliability dimension score, and operation rationality dimension score are weighted and summed according to the preset comprehensive weights to obtain the comprehensive score of the candidate gripping point in the current frame image.

5. The visual-guided precision operation method for the pineapple seedling clamping robotic arm according to claim 4, characterized in that: Based on the comprehensive score sequence, the temporal stability score of each candidate gripping point is calculated, and based on the changing trend of the comprehensive score sequence, the future sustainability score of each candidate gripping point is calculated. The specific steps are as follows: For each candidate gripping point, the comprehensive score of the candidate gripping point in multiple consecutive frames within a preset time window is extracted and arranged in chronological order of the frames to form a corresponding comprehensive score sequence. For a comprehensive score sequence, the variance of all comprehensive scores in the sequence is calculated, and the variance is matched with multiple preset variance intervals. The time series stability score corresponding to the variance interval that matches the variance is used as the time series stability score of the candidate clamping point. For the comprehensive score sequence, the least squares method is used to fit a linear trend line of the comprehensive score changing over time, the slope of the linear trend line is obtained, and the slope is matched with multiple preset slope intervals. The future sustainability score corresponding to the slope interval that matches the slope is used as the future sustainability score of the candidate clamping point.

6. The visual-guided precision operation method for the pineapple seedling clamping robotic arm according to claim 5, characterized in that: The comprehensive score, temporal stability score, and future sustainability score of each candidate clamping point in the last frame of the time window are weighted and fused. The specific steps are as follows: The temporal stability score of each candidate clamping point is normalized to obtain a normalized temporal stability value, and the normalized temporal stability value is used as a discount factor. The first intermediate score is obtained by multiplying the comprehensive score of each candidate clamping point in the last frame image by the corresponding discount factor; The future sustainability score of each candidate clamping point is normalized to obtain a future sustainability normalized value, and the future sustainability normalized value is used as a gain factor. The first intermediate score is multiplied by the corresponding gain factor to obtain the final comprehensive score of the candidate clamping point.

7. A vision-guided precision operation device for a pineapple seedling gripping robotic arm, characterized in that: The operating device is used to perform the vision-guided precision operation method of the pineapple seedling clamping robotic arm according to any one of claims 1-6, including: The feature extraction module is used to acquire pineapple seedling images, perform target segmentation on the pineapple seedling images to obtain pineapple seedling regions, extract skeleton curves and contour width curves based on the pineapple seedling regions, and calculate local maxima of curvature based on the skeleton curves to form a set of curvature extrema points. The candidate point generation module is used to calculate local maxima and local minima based on the contour width curve, form a set of width extrema points, and use the union of the curvature extrema point set and the width extrema point set as a set of candidate clamping points. The scoring calculation module is used to construct a time window containing the current frame and several frames of images before the current frame. For each candidate clamping point in each frame of the time window, feature values ​​on multiple preset evaluation dimensions are extracted, and the feature values ​​are converted into corresponding dimension scores to obtain the comprehensive score of each candidate clamping point in each frame of the image. The temporal evaluation module is used to construct a comprehensive score sequence corresponding to each candidate clamping point based on the comprehensive score of each candidate clamping point in each frame image within the time window, calculate the temporal stability score of each candidate clamping point based on the comprehensive score sequence, and calculate the future sustainability score of each candidate clamping point based on the changing trend of the comprehensive score sequence. The weighted fusion module is used to perform weighted fusion of the comprehensive score, temporal stability score and future sustainability score of each candidate gripping point in the last frame of the time window to obtain the final comprehensive score of each candidate gripping point, and to determine the candidate gripping point with the highest final comprehensive score as the optimal candidate gripping point, and control the gripping robot arm to perform gripping operations based on the optimal candidate gripping point.

Images