Tea leaf picking time prediction method and system based on tea leaf image analysis

Abstract

This invention relates to the field of image analysis technology, specifically to a method and system for predicting tea picking time based on tea image analysis. The method involves determining the optimal picking frame for tea buds and labeling each frame with a time difference tag between the optimal picking frame and the frame. For each frame in the training dataset, the foreground region of the tea buds is extracted, and bud stem contour segments are cropped. The rate of curvature change on the bud stem contour segments is calculated as a mechanical compensation feature characterizing the degree of lignification of the bud stem. The angle between the principal axis vector of the bud stem and the midrib vector of each leaf is calculated to obtain the leaf unfolding angle feature. A mapping relationship between the remaining time to the optimal picking time is established by splicing the mechanical compensation feature, the leaf unfolding angle feature, and the time difference tag. Real-time images of the tea garden to be tested are acquired, and the mechanical compensation feature and leaf unfolding angle feature of the tea bud samples are extracted. The picking time prediction result is output through the mapping relationship, thus improving the accuracy of tea picking timing prediction.

Description

Technical Field

[0001] This invention relates to the field of image analysis technology, specifically to a method and system for predicting tea picking time based on tea image analysis. Background Technology

[0002] Accurately determining the timing of tea picking is crucial to the quality and economic value of tea. Tea buds undergo continuous growth stages, including germination, leaf unfolding, and maturation. Significant differences exist in the internal composition, leaf expansion, and stem structure of buds at different stages. Picking too early or too late will lead to a decline in tea quality. Traditional tea picking relies on experienced tea pickers who visually observe the leaf unfolding shape and bud size, combined with feeling the firmness of the stem by touch, to determine the optimal picking time. This process is highly dependent on accumulated personal experience and is difficult to standardize.

[0003] Existing methods typically use morphological indicators such as leaf color, bud length, and leaf area, which can be directly obtained from images, as the basis for determining the harvesting time. However, during the growth process, the stems of tea buds undergo significant changes in mechanical properties as the degree of cell lignification deepens. The stems gradually become stiffer from their initial softness and flexibility. This change in internal tissue structure is not visually apparent and cannot be effectively perceived based solely on visible morphological features such as leaf unfolding angle and color distribution in images. Consequently, during the critical time window when the appearance of tea buds has stabilized but the degree of lignification in the stems still varies, it is impossible to make reliable harvesting time determinations, resulting in insufficient accuracy in harvesting time prediction. Summary of the Invention

[0004] (1) Technical problems to be solved The purpose of this invention is to provide a method and system for predicting tea picking time based on tea image analysis, which solves the problem that relying solely on visible morphological features to determine the picking time is insufficient because it cannot perceive the different degrees of lignification in the tea buds and stems, making it difficult to accurately determine the picking time of tea.

[0005] (2) Technical solution To achieve the above objectives, on the one hand, the present invention provides a method for predicting tea picking time based on tea image analysis, the method comprising: S1. Collect multiple time-series images of historical tea buds throughout their complete growth cycle as a training dataset; determine the optimal picking frame from the multiple time-series images, and label each frame of the multiple time-series images with the time difference between the optimal picking frame and the optimal picking frame.

[0006] S2. Perform tea bud instance segmentation on each frame image in the training dataset and extract the tea bud foreground region; extract the skeleton of the tea bud foreground region to obtain the tea bud skeleton line, and detect and locate the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the base of each leaf on the tea bud skeleton line.

[0007] S3. Based on the relative positional relationship between the base point of the bud stem and the key points of the leaf base of each leaf, extract the bud stem outline segment from the outline of the tea bud foreground area, and calculate the curvature change rate on the bud stem outline segment as a mechanical compensation feature characterizing the degree of lignification of the bud stem.

[0008] S4. Based on the basal point of the bud stem and the key points of the leaf tip and leaf base of each leaf, construct the bud stem principal axis vector representing the direction of the bud stem and the leaf midrib vector representing the direction of the midrib of each leaf, and calculate the angle between the bud stem principal axis vector and the midrib vector of each leaf to obtain the leaf unfolding angle feature.

[0009] S5. The mechanical compensation features and leaf unfolding angle features are concatenated into a multi-dimensional feature vector. The time difference label is used as the regression target. A mapping relationship is established from the multi-dimensional feature vector to the remaining time from the optimal picking time based on the training dataset. Real-time images of the tea garden to be tested are obtained and the mechanical compensation features and leaf unfolding angle features of the tea bud samples are extracted. The picking time prediction result is output through the mapping relationship.

[0010] Furthermore, the method for determining the optimal capture frame among the multiple time-series images includes: The collected historical tea buds were sorted into a time-series image sequence by the order of their collection time. The leaf unfolding angle feature was extracted from each frame of the time-series image sequence to obtain the leaf unfolding angle feature value sequence.

[0011] Calculate the first-order difference between two adjacent frames in the leaf angle feature value sequence to obtain the leaf angle change rate sequence; take the frame corresponding to the position where the change rate value in the leaf angle change rate sequence first drops to less than the preset change rate threshold as the starting frame when the leaf angle tends to stabilize.

[0012] Starting from the initial frame, a preset number of subsequent images are continuously taken in the time-increasing direction to form a stable window. The arithmetic mean of the leaf angle feature values ​​of all frames in the stable window is calculated as the average leaf angle during the stable period, and the standard deviation of the leaf angle feature values ​​of all frames in the stable window is calculated as the fluctuation amplitude during the stable period.

[0013] When the fluctuation amplitude during the stable period is less than the preset fluctuation threshold, the frame image in the middle of the time within the stable window is marked as the candidate best capture frame; the mechanical compensation feature of the candidate best capture frame is obtained. If the mechanical compensation feature is less than the preset mechanical compensation threshold, the candidate best capture frame is determined as the best capture frame; otherwise, backtracking along the decreasing time direction to the first frame image before the stable window, the first frame image is used as the new candidate best capture frame, and the mechanical compensation feature of the new candidate best capture frame is compared with the preset mechanical compensation threshold again, until a frame image with a mechanical compensation feature less than the preset mechanical compensation threshold is found as the best capture frame.

[0014] Furthermore, the method for calculating the rate of change of curvature on the bud-stem contour segment as a mechanical compensatory feature characterizing the degree of lignification of the bud-stem includes: On the outline segment of the bud stem extracted from the outline of the tea bud foreground region, a sequence of outline sampling points with equal arc length intervals is extracted along the extension direction of the bud stem; a local arc is fitted according to the position coordinates of every three adjacent sampling points in the outline sampling point sequence, and the reciprocal of the radius of the local arc is used as the curvature value of the intermediate sampling point to obtain a discrete curvature sequence along the bud stem axis.

[0015] The discrete curvature sequence is subjected to a differential operation along the bud stem axis to obtain the curvature difference value between adjacent sampling points, and the curvature difference value is divided by the arc length interval between adjacent sampling points to obtain the curvature change rate sequence along the bud stem axis.

[0016] The maximum absolute value of the rate of change of curvature is extracted from the curvature change rate sequence as a mechanical compensation feature.

[0017] Furthermore, the method for extracting a sequence of contour sampling points with equal arc length intervals along the extension direction of the bud stem on the bud stem contour segment extracted from the contour of the tea bud foreground region includes: After extracting the bud and stem outline segment from the outline of the tea bud foreground area, curve fitting is performed on the bud and stem outline segment to obtain a continuous parametric curve expression of the bud and stem outline; on the continuous parametric curve expression of the bud and stem outline, the total arc length of the bud and stem outline is calculated with the bud and stem base point as the starting point and the connection point between the bud and stem and the leaf as the ending point.

[0018] The total arc length of the bud and stem outline is divided into a preset number of arc length intervals. At the endpoint of each arc length interval, the coordinate points on the continuous parameterized curve expression of the bud and stem outline are extracted to form a sequence of contour sampling points with equal arc length intervals extracted along the extension direction of the bud and stem.

[0019] Furthermore, the method for curve fitting the bud-stem contour segment to obtain a continuous parametric curve expression for the bud-stem contour includes: The coordinate sequence of all contour pixels on the bud and stem contour segment is extracted, and the coordinate sequence is subjected to sliding window averaging filter to eliminate jagged noise at the image segmentation boundary, resulting in a smoothed contour pixel coordinate sequence.

[0020] The smoothed contour pixel coordinate sequence is interpolated and fitted using a cubic spline interpolation method to generate a continuous parameterized curve expression for the bud and stem contour with arc length as the independent variable. The fitting deviation at each contour pixel is calculated based on the continuous parameterized curve expression of the bud and stem contour. If the fitting deviation at any contour pixel exceeds a preset deviation threshold, the contour pixel is removed and cubic spline interpolation is performed again until the fitting deviation at all retained contour pixels does not exceed the preset deviation threshold, thus obtaining the continuous parameterized curve expression of the bud and stem contour.

[0021] Furthermore, the method for extracting the maximum absolute value of the rate of change of curvature from the curvature change rate sequence as a mechanical compensation feature includes: Extreme points are detected along the bud stem axis of the curvature change rate sequence, and all local extreme points in the curvature change rate sequence whose absolute value of curvature change rate is greater than the absolute value of curvature change rate of adjacent sampling points are extracted.

[0022] A local curvature change rate window is formed by taking a preset number of sampling points forward and backward along the bud stem axis, with each local extreme point as the center. The local standard deviation of the absolute value of the curvature change rate is calculated within the local curvature change rate window. If the local standard deviation is less than the preset standard deviation threshold, it is determined that there is a change in the actual bending stiffness at the bud stem contour position corresponding to the local extreme point due to the abrupt change in the degree of lignification.

[0023] If the local standard deviation is greater than the preset standard deviation threshold, the bud and stem contour position corresponding to the local extreme point is determined to be a pseudo-bending change point and is removed.

[0024] From the local extreme points corresponding to the retained true bending stiffness changes, the maximum absolute value of the rate of change of curvature is selected as the mechanical compensation feature.

[0025] Furthermore, the method of selecting the maximum absolute value of the rate of change of curvature from the local extreme points corresponding to the retained true bending stiffness changes as the mechanical compensation feature includes: Obtain all local extreme points corresponding to the actual bending stiffness changes, and record the arc length coordinates of each local extreme point corresponding to the actual bending stiffness change on the bud-stem contour segment relative to the bud-stem base point.

[0026] Based on the arc length coordinates of the local extreme points corresponding to each actual change in bending stiffness, local extreme points with adjacent arc length coordinates and a spacing less than a preset spacing threshold are merged into the same bending change segment to obtain at least one bending change segment.

[0027] Calculate the segment average of the absolute values ​​of the rate of curvature change at all local extreme points within each bending change segment, and use the segment average as the bending abrupt change intensity of the bending change segment; select the bending change segment with the largest bending abrupt change intensity from all bending change segments, and within the bending change segment with the largest bending abrupt change intensity, select the maximum absolute value of the rate of curvature change as the mechanical compensation characteristic.

[0028] Furthermore, the method for establishing the mapping relationship from the multidimensional feature vector to the remaining time until the optimal harvesting time based on the training dataset includes: The mechanical compensation features and leaf unfolding angle features extracted from each historical tea bud image in the training dataset are concatenated to form a multidimensional feature vector representing the current growth state of the tea buds. At the same time, the time difference label corresponding to each frame image is numerically normalized to obtain the normalized time difference label. The mean squared error loss function is used to measure the difference between the prediction output of the initial mapping model and the normalized time difference label.

[0029] An initial mapping model based on gradient boosting regression tree is constructed, with the multidimensional feature vector as the input feature of the initial mapping model and the normalized time difference label as the output target of the initial mapping model.

[0030] Calculate the prediction residual of the current initial mapping model for each training sample. The prediction residual is the normalized time difference label minus the prediction output of the current initial mapping model. Fit a new regression tree with the multidimensional feature vector as the independent variable and the prediction residual as the target variable, and update the initial mapping model. Stop training when the mean squared error loss function no longer decreases after a preset number of consecutive rounds. Obtain the gradient boosting regression tree model as a mapping relationship between the multidimensional feature vector and the remaining time to the optimal picking time.

[0031] Furthermore, the method for acquiring real-time images of the tea garden to be tested and extracting the mechanical compensation features and leaf unfolding angle features of tea bud samples, and then outputting the harvesting time prediction result through the mapping relationship includes: The tea garden to be predicted is divided into several grid units according to a preset grid size; real-time images are acquired for each grid unit, and a preset number of tea bud samples are randomly selected from the real-time images. Mechanical compensation features and leaf unfolding angle features are extracted for each tea bud sample, and the remaining time prediction value of each tea bud sample from the optimal picking time is calculated through the mapping relationship; the arithmetic mean of the remaining time prediction values ​​of all tea bud samples in the same grid unit is calculated as the picking time prediction result and output.

[0032] On the other hand, based on the same inventive concept, this invention also provides a tea picking time prediction system based on tea image analysis, the system comprising: The training dataset construction and labeling module is used to collect multiple time-series images of historical sample tea buds during their complete growth cycle as the training dataset; the optimal picking frame is determined from the multiple time-series images, and each frame in the multiple time-series images is labeled with a time difference label between the optimal picking frame and the optimal picking frame.

[0033] The tea bud key point detection and localization module is used to segment tea bud instances in each frame of the training dataset and extract the tea bud foreground region; extract the skeleton of the tea bud foreground region to obtain the tea bud skeleton line; and detect and locate the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the base of each leaf on the tea bud skeleton line.

[0034] The mechanical compensation feature extraction module is used to extract the bud and stem outline segment from the outline of the tea bud foreground area based on the relative positional relationship between the bud and stem base point and the leaf base key point of each leaf, and calculate the curvature change rate on the bud and stem outline segment as a mechanical compensation feature characterizing the degree of lignification of the bud and stem.

[0035] The leaf unfolding angle feature extraction module is used to construct the bud stem principal axis vector representing the direction of the bud stem and the leaf midrib vector representing the direction of the midrib of each leaf based on the bud stem base point and the leaf tip key point and leaf base key point of each leaf, and calculate the angle between the bud stem principal axis vector and the midrib vector of each leaf to obtain the leaf unfolding angle feature.

[0036] The picking time prediction output module is used to concatenate the mechanical compensation features and leaf unfolding angle features into a multi-dimensional feature vector, take the time difference label as the regression target, and establish a mapping relationship from the multi-dimensional feature vector to the remaining time from the optimal picking time based on the training dataset; acquire real-time images of the tea garden to be tested and extract the mechanical compensation features and leaf unfolding angle features of the tea bud samples, and output the picking time prediction result through the mapping relationship.

[0037] (3) Beneficial effects Compared with the prior art, the beneficial effects of the present invention are: 1. By collecting multiple time-series images of tea buds from historical samples, the optimal picking frame is determined and labeled with a time difference tag. Combined with the extraction and analysis of the mechanical compensation characteristics and leaf unfolding angle characteristics of tea buds, a mapping relationship is established from multi-dimensional feature vectors to the remaining time before the optimal picking time. This allows for more accurate capture of subtle changes in the growth process of tea buds, thereby accurately predicting the picking time, ensuring that tea is picked in its best condition, and effectively improving tea quality.

[0038] 2. When calculating the mechanical compensation features, the contour sampling point sequence is extracted with equal arc length intervals, and curve fitting, difference operation and other processing are performed. The maximum absolute value is extracted from the curvature change rate sequence as the mechanical compensation feature. At the same time, the leaf unfolding angle feature is accurately calculated, which can more accurately represent the growth state of tea buds, reduce the prediction error caused by inaccurate feature extraction, and greatly improve the reliability of the harvest time prediction. Attached Figure Description

[0039] Figure 1 This is a flowchart of the tea picking time prediction method based on tea image analysis according to the present invention.

[0040] Figure 2 This is a schematic diagram of the module composition of the tea picking time prediction system based on tea image analysis of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] Example 1: As Figure 1 As shown, this embodiment provides a method for predicting tea picking time based on tea image analysis. The method includes: S1. Collect multiple time-series images of historical tea buds throughout their complete growth cycle as a training dataset; determine the optimal picking frame from the multiple time-series images.

[0043] The method for determining the optimal capture frame from the multiple time-series images includes: The collected historical tea buds were sorted into a time sequence by the order of collection time. The leaf unfolding angle features were extracted from each frame of the time sequence to obtain a leaf unfolding angle feature value sequence. Calculate the first-order difference between two adjacent frames in the leaf angle feature value sequence to obtain the leaf angle change rate sequence; take the frame corresponding to the position where the change rate value in the leaf angle change rate sequence first drops to less than the preset change rate threshold as the starting frame when the leaf angle tends to stabilize. Starting from the initial frame, a preset number of subsequent images are continuously taken in the time-increasing direction to form a stable window. The arithmetic mean of the leaf angle feature values ​​of all frames in the stable window is calculated as the average leaf angle during the stable period, and the standard deviation of the leaf angle feature values ​​of all frames in the stable window is calculated as the fluctuation amplitude during the stable period. When the fluctuation amplitude during the stable period is less than the preset fluctuation threshold, the frame image in the middle of the time within the stable window is marked as the candidate best capture frame; the mechanical compensation feature of the candidate best capture frame is obtained. If the mechanical compensation feature is less than the preset mechanical compensation threshold, the candidate best capture frame is determined as the best capture frame; otherwise, backtracking along the decreasing time direction to the first frame image before the stable window, the first frame image is used as the new candidate best capture frame, and the mechanical compensation feature of the new candidate best capture frame is compared with the preset mechanical compensation threshold again, until a frame image with a mechanical compensation feature less than the preset mechanical compensation threshold is found as the best capture frame.

[0044] For example, representative historical tea buds are selected from a tea garden. Throughout the entire growth cycle, a fixed-focus camera continuously tracks and photographs the same tea bud, capturing multiple frames at fixed time intervals, such as one frame every two hours, forming a time-series image sequence. The acquisition time interval between adjacent frames remains fixed throughout the entire acquisition cycle; that is, continuous shooting is performed at equal time intervals. The fixed time interval is the basis for subsequent calculations of the leaf unfolding angle change rate sequence and time difference labels, because the value of each element in the leaf unfolding angle change rate sequence directly corresponds to the change in leaf unfolding angle within a unit time interval. The time difference label is also converted into a time distance in hours by multiplying the number of frames by the fixed time interval. During actual acquisition, the consistency of the time interval between frames must be strictly ensured. The acquisition time of each frame is recorded to ensure that the timestamp information of each frame in the time-series image sequence is complete and traceable. After obtaining the leaf unfolding angle feature value sequence, the first-order difference of the leaf unfolding angle between two adjacent frames in the sequence is calculated. This is achieved by subtracting the leaf unfolding angle feature value of the (i-1)th frame from the leaf unfolding angle feature value of the i-th frame, resulting in the leaf unfolding angle change rate sequence. Starting from the first element of the leaf unfolding angle change rate sequence, the absolute value of each change rate value is compared with a preset change rate threshold. The position where the absolute value of the change rate first drops below the preset change rate threshold is identified, and the subsequent frame corresponding to this position is determined as the starting frame where the leaf unfolding angle tends to stabilize. The absolute value is used here to simultaneously cover the case where the leaf unfolding angle slightly retracts (negative change rate), ensuring that the judgment of the change amplitude is not affected by direction. In the early stages of tea bud growth, the leaf unfolding speed is relatively fast, and the leaf unfolding angle change rate value is large. As the leaf unfolding stabilizes, the change rate value gradually decreases and stabilizes at a low level. Therefore, the first drop of the absolute value of the change rate below the preset change rate threshold corresponds to the turning point where the leaf unfolding angle transitions from a rapid growth phase to a stable phase. The preset change rate threshold can be determined through historical sample statistics based on different tea varieties and growing environments. For example, 20% of the median of the leaf unfolding angle change rate sequence in historical samples can be used as a reference benchmark. In practical applications, the value range is usually between 1° / frame and 3° / frame.

[0045] The preset number of frames is determined based on the acquisition frequency and the growth rate of the tea variety to ensure that the stable window covers the complete period during which the tea buds and leaves tend to stabilize. For example, if one frame is acquired every 2 hours, the preset number of frames can be 12, corresponding to a stable observation period of approximately 24 hours. The arithmetic mean of the leaf unfolding angle feature values ​​of all frames within the stable window is calculated as the average leaf unfolding angle during the stable period, and the standard deviation of the leaf unfolding angle feature values ​​of all frames within the stable window is calculated as the fluctuation range during the stable period.

[0046] When the fluctuation amplitude during the stable period is less than the preset fluctuation threshold, the tea bud leaf unfolding angle is considered to have entered a stable period. The preset fluctuation threshold is used to measure whether the dispersion of the leaf unfolding angle within the stable window meets the stability requirements. It can be determined based on the statistical distribution of the leaf unfolding angle during the stable period in historical samples. For example, the 75th percentile of the standard deviation distribution of the leaf unfolding angle during the stable period in historical samples can be used as the preset fluctuation threshold, with a typical value range between 0.5° and 2°. When the fluctuation amplitude during the stable period calculated within the stable window constructed based on the current starting frame is not less than the preset fluctuation threshold, it indicates that the leaf unfolding angle within the stable window starting from the current starting frame has not yet truly entered a stable state. At this time, the frame after the current starting frame in the time-series image sequence is taken as the new starting frame, the stable window is reconstructed, and the fluctuation amplitude during the stable period is calculated again and compared with the preset fluctuation threshold. This process is repeated frame by frame along the time-increasing direction until a stable window that satisfies the condition that the fluctuation amplitude during the stable period is less than the preset fluctuation threshold is found. After the stability condition is met, the frame image with the time in the middle within the stable window is marked as the candidate best picking frame.

[0047] If the mechanical compensation characteristics of a candidate best picking frame are less than a preset mechanical compensation threshold, then the candidate best picking frame is determined as the best picking frame; otherwise, it indicates that the lignification of the tea bud stem has not yet reached a suitable state for picking. In this case, the process is to backtrack along the decreasing time direction to the first frame image before the stable window, and use this frame image as a new candidate best picking frame. The mechanical compensation characteristics of the new candidate best picking frame are compared with the preset mechanical compensation threshold again. This backtracking comparison is repeated until a frame image with mechanical compensation characteristics less than the preset mechanical compensation threshold is found, and this frame image is determined as the best picking frame. The preset mechanical compensation threshold is determined based on the agronomic standards and picking process requirements of the tea variety, through manual annotation combined with statistical analysis of mechanical compensation characteristics. For example, experienced tea pickers score the stem softness of each frame image in the historical samples, and the upper boundary of the mechanical compensation characteristic distribution interval corresponding to the score reaching the "suitable for picking" level is used as the preset mechanical compensation threshold.

[0048] Each frame in the multi-frame time-series image is labeled with a time difference tag between itself and the optimal harvesting frame. The time difference tag is converted into a time distance in hours by multiplying the number of frames by a fixed time interval. For frames before the optimal harvesting frame, the time difference tag is negative, and the absolute value indicates how many hours are left until the optimal harvesting time. For frames after the optimal harvesting frame, the time difference tag is positive, indicating how many hours have passed since the harvesting opportunity. The time difference tag for the optimal harvesting frame itself is 0.

[0049] S2. Perform tea bud instance segmentation on each frame image in the training dataset and extract the tea bud foreground region; extract the skeleton of the tea bud foreground region to obtain the tea bud skeleton line, and detect and locate the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the base of each leaf on the tea bud skeleton line.

[0050] For example, each frame of the training dataset is segmented into tea bud instances to separate the tea bud foreground region from the complex tea garden background. Instance segmentation can be performed using a deep learning-based instance segmentation model, such as Mask R-CNN or a similar architecture. The segmentation result represents the pixel range of the tea bud foreground region in the form of a binary mask image. In the mask image, the region with a pixel value of 1 corresponds to the tea bud foreground, and the region with a pixel value of 0 corresponds to the background.

[0051] Skeleton extraction is performed on the foreground region of the tea buds. This involves morphological thinning of the binary mask image of the foreground region, compressing it into a single-pixel-width skeleton line that preserves the overall topological structure of the tea bud. On this skeleton line, key points are detected and located by analyzing topological features such as endpoints and bifurcation points: the single-pixel endpoint at the very top of the skeleton line is designated as the bud tip key point; the single-pixel endpoint at the very bottom of the skeleton line near the stem base is designated as the bud-stem base point; for each leaf, the endpoint where the leaf skeleton line extends to the leaf outline edge is designated as the leaf tip key point, and the bifurcation point between the leaf skeleton line and the bud-stem skeleton line is designated as the leaf base key point.

[0052] S3. Based on the relative positional relationship between the base point of the bud stem and the key points of the leaf base of each leaf, extract the bud stem outline segment from the outline of the tea bud foreground area, and calculate the curvature change rate on the bud stem outline segment as a mechanical compensation feature characterizing the degree of lignification of the bud stem.

[0053] The method for calculating the rate of curvature change on the bud-stem contour segment as a mechanical compensatory feature characterizing the degree of lignification of the bud-stem includes: On the outline segment of the bud stem extracted from the outline of the foreground region of the tea bud, a sequence of outline sampling points with equal arc length intervals is extracted along the extension direction of the bud stem.

[0054] The method for extracting a sequence of contour sampling points with equal arc length intervals along the extension direction of the bud stem on the bud stem contour segment extracted from the contour of the tea bud foreground region includes: After extracting the bud and stem outline segment from the outline of the tea bud foreground area, curve fitting is performed on the bud and stem outline segment to obtain a continuous parametric curve expression of the bud and stem outline.

[0055] The method for curve fitting the bud-stem contour segment to obtain a continuous parametric curve expression for the bud-stem contour includes: The coordinate sequence of all contour pixels on the bud and stem contour segment is extracted, and the coordinate sequence is subjected to sliding window averaging filter to eliminate jagged noise at the image segmentation boundary, resulting in a smoothed contour pixel coordinate sequence.

[0056] The smoothed contour pixel coordinate sequence is interpolated and fitted using a cubic spline interpolation method to generate a continuous parameterized curve expression for the bud and stem contour with arc length as the independent variable. The fitting deviation at each contour pixel is calculated based on the continuous parameterized curve expression of the bud and stem contour. If the fitting deviation at any contour pixel exceeds a preset deviation threshold, the contour pixel is removed and cubic spline interpolation is performed again until the fitting deviation at all retained contour pixels does not exceed the preset deviation threshold, thus obtaining the continuous parameterized curve expression of the bud and stem contour.

[0057] For example, mechanical compensation features are used to indirectly reflect changes in bending stiffness caused by the degree of lignification of the bud stem. When the degree of lignification of the cells in the tea bud stem increases, the local bending stiffness of the stem undergoes abrupt changes. Under the action of external forces (such as gravity and wind), the curvature distribution of the stem profile will show extreme values ​​of the rate of change of local curvature at the location of the abrupt change in bending stiffness. It is by utilizing this mechanical compensation phenomenon that feature quantities characterizing the degree of lignification are extracted from the rate of change of curvature of the stem profile.

[0058] On the outline of the tea bud foreground region, the starting point for cropping is the corresponding position of the bud stem base point on the outline, and the ending point is the corresponding position of the leaf base key point closest to the bud stem base point on the outline. The outline pixel sequence within this range is extracted to obtain the bud stem outline segment. After extracting the coordinate sequence of all outline pixels on the bud stem outline segment, a sliding window averaging filter is applied to the coordinate sequence to eliminate jagged noise caused by pixelation at the image segmentation boundary. The window length of the sliding window can be 5 to 9 pixels. The arithmetic mean of the x-coordinate and y-coordinate of all pixels within the window is taken, and the average value replaces the coordinate of the window center point. This process is gradually slidable to smooth the entire coordinate sequence, resulting in a smoothed outline pixel coordinate sequence. Cubic spline interpolation is then used to fit the smoothed outline pixel coordinate sequence, generating a continuous parametric curve expression for the bud stem outline with the arc length parameter as the independent variable. Specifically, the Euclidean distance between adjacent contour pixels is calculated and accumulated sequentially to obtain the arc length parameter value corresponding to each pixel. Then, a cubic spline interpolation function x=f(s) is constructed with the arc length parameter as the independent variable and the horizontal coordinate of the pixel as the dependent variable, and a cubic spline interpolation function y=g(s) is constructed with the arc length parameter as the independent variable and the vertical coordinate of the pixel as the dependent variable, where s is the arc length parameter. The two interpolation functions together constitute the continuous parameterized curve expression of the bud and stem contour.

[0059] After initial fitting, the fitting deviation at each contour pixel is calculated, which is the Euclidean distance between the original pixel coordinates and the calculated coordinates of the parametric curve at the corresponding arc length parameter. The calculation formula is as follows: , where (x i ,y i ) represents the original coordinates of the i-th contour pixel, s i This represents the arc length parameter value corresponding to the pixel. If the fitting deviation at any contour pixel exceeds a preset deviation threshold, that contour pixel is identified as an outlier and removed. Then, cubic spline interpolation is performed again on the coordinate sequence of the remaining contour pixels. This process is iterated until the fitting deviation at all remaining contour pixels does not exceed the preset deviation threshold, ultimately yielding a continuous parametric curve expression for the bud / stem contour. The preset deviation threshold can be determined based on the image resolution, for example, by 1 to 2 pixels.

[0060] On the continuous parametric curve expression of the bud-stem outline, the total arc length of the bud-stem outline is calculated with the bud-stem base point as the starting point and the connection point between the bud-stem and the leaf as the ending point.

[0061] The total arc length of the bud and stem outline is divided into a preset number of arc length intervals. At the endpoint of each arc length interval, the coordinate points on the continuous parameterized curve expression of the bud and stem outline are extracted to form a sequence of contour sampling points with equal arc length intervals extracted along the extension direction of the bud and stem.

[0062] By fitting a local arc to the position coordinates of every three adjacent sampling points in the contour sampling point sequence, and using the reciprocal of the radius of the local arc as the curvature value of the intermediate sampling point, a discrete curvature sequence along the bud stem axis is obtained.

[0063] The discrete curvature sequence is subjected to a differential operation along the bud stem axis to obtain the curvature difference value between adjacent sampling points, and the curvature difference value is divided by the arc length interval between adjacent sampling points to obtain the curvature change rate sequence along the bud stem axis.

[0064] For example, on the continuous parametric curve expression of the bud and stem contour, starting from the bud and stem base point and ending at the point where the bud and stem connect to the leaf (i.e., the corresponding position of the leaf base key point closest to the bud and stem base point on the contour), the total arc length L of the bud and stem contour is calculated by integrating the parametric curve within the corresponding arc length parameter interval. The total arc length L is divided into a preset number of N arc length intervals, each with a length of Δs = L / N. At the endpoint of each arc length interval, the coordinate values ​​are calculated from the parametric curve expression at the corresponding arc length parameter value, resulting in N+1 contour sampling points with equal arc length intervals, forming a sequence of contour sampling points along the extension direction of the bud and stem. The value of the preset number N needs to ensure that the arc length interval between adjacent sampling points is small enough to accurately describe the curvature change, while avoiding N being too large to amplify noise. In practice, it can be determined according to the ratio of the total arc length of the bud and stem to the image resolution, for example, N = 20 to 50.

[0065] For each three adjacent sampling points in the contour sampling point sequence, a local arc is fitted using these three points. The radius R of the arc is calculated, and the reciprocal of the radius, 1 / R, is used as the curvature value κ at the intermediate sampling point, resulting in a discrete curvature sequence {κ1,κ2,...,κ} along the shoot axis. n-1 The specific calculation method for the local circular arc fitted by three points is as follows: Let the coordinates of three adjacent sampling points be (x1, y1), (x2, y2), and (x3, y3). Solve for the coordinates of the center (a, b) and the radius R by solving the system of equations of the circle passing through the three points. That is, let (x1, y1), (x2, y2), and (x3, y3) be the coordinates of the center (a, b) and the radius R. i -a)²+(y i -b)²=R² holds true for i=1,2,3. After solving the system of equations, the reciprocal of R, 1 / R, is taken as the curvature value of the intermediate sampling point. During the calculation, there is a degradation case where three points are collinear, meaning the three sampling points are approximately on the same straight line. In this case, the radius of the arc tends to infinity, and the corresponding curvature value approaches zero. In actual calculations, the degradation case is identified by judging whether the area of ​​the triangle formed by the three points is less than a preset area threshold. The area of ​​the triangle can be calculated using the determinant formula: S=|(x2-x1)(y3-y1)-(x3-x1)(y2-y1)| / 2, and the preset area threshold can be 0.1 square pixels. When collinearity of three points is detected, the curvature value of the intermediate sampling point is directly assigned to zero instead of solving for the radius of the arc, to avoid division by zero errors or large values ​​interfering with subsequent curvature change rate calculations. A first-order difference operation is performed on the discrete curvature sequence along the bud stem axis to calculate the curvature difference value Δκ between adjacent sampling points. i =κ i+1 -κ i and the curvature difference value Δκ i Dividing by the fixed arc length interval Δs between adjacent sampling points yields the rate of curvature change r. i =Δκ i / Δs, thus obtaining the curvature change rate sequence {r1,r2,...,rn-2}

[0066] The maximum absolute value of the rate of change of curvature is extracted from the curvature change rate sequence as a mechanical compensation feature.

[0067] The method for extracting the maximum absolute value of the rate of change of curvature from the curvature change rate sequence as a mechanical compensation feature includes: Extreme points are detected along the bud stem axis of the curvature change rate sequence, and all local extreme points in the curvature change rate sequence whose absolute value of curvature change rate is greater than the absolute value of curvature change rate of adjacent sampling points are extracted.

[0068] A local curvature change rate window is formed by taking a preset number of sampling points forward and backward along the bud stem axis, with each local extreme point as the center. The local standard deviation of the absolute value of the curvature change rate is calculated within the local curvature change rate window. If the local standard deviation is less than the preset standard deviation threshold, it is determined that there is a change in the actual bending stiffness at the bud stem contour position corresponding to the local extreme point due to the abrupt change in the degree of lignification.

[0069] If the local standard deviation is greater than the preset standard deviation threshold, the bud and stem contour position corresponding to the local extreme point is determined to be a pseudo-bending change point and is removed.

[0070] For example, extreme point detection is performed on the curvature change rate sequence, and the absolute value of the curvature change rate |r| in the sequence is extracted. i |simultaneously greater than the absolute value of the rate of change of curvature at adjacent sampling points|r i-1 |and|r i+1 |All local extreme points. For each local extreme point, a local curvature change rate window (2m+1 points in total) is formed by taking a preset number of sampling points m along the shoot axis centered on that extreme point, both forward and backward. The local standard deviation of the absolute values ​​of all curvature change rates within the window is calculated. If the local standard deviation is less than the preset standard deviation threshold, the shoot contour position corresponding to the extreme point is determined to have a change in true bending stiffness caused by a sudden change in lignification degree and is retained; if the local standard deviation is greater than the preset standard deviation threshold, the position corresponding to the extreme point is determined to be a pseudo-bending change point (i.e., a false extreme point caused by image noise or segmentation error) and is discarded. The preset number of sampling points m can be 3 to 5, and the preset standard deviation threshold can be determined by statistical analysis based on the distribution of local standard deviations corresponding to true lignification positions and pseudo-noise positions in historical samples.

[0071] From the local extreme points corresponding to the retained true bending stiffness changes, the maximum absolute value of the rate of change of curvature is selected as the mechanical compensation feature.

[0072] The method of selecting the maximum absolute value of the rate of change of curvature from the local extreme points corresponding to the retained true bending stiffness changes as the mechanical compensation feature includes: Obtain all local extreme points corresponding to the actual bending stiffness changes, and record the arc length coordinates of each local extreme point corresponding to the actual bending stiffness change on the bud-stem contour segment relative to the bud-stem base point.

[0073] Based on the arc length coordinates of the local extreme points corresponding to each actual change in bending stiffness, local extreme points with adjacent arc length coordinates and a spacing less than a preset spacing threshold are merged into the same bending change segment to obtain at least one bending change segment.

[0074] Calculate the segment average of the absolute values ​​of the rate of curvature change at all local extreme points within each bending change segment, and use the segment average as the bending abrupt change intensity of the bending change segment; select the bending change segment with the largest bending abrupt change intensity from all bending change segments, and within the bending change segment with the largest bending abrupt change intensity, select the maximum absolute value of the rate of curvature change as the mechanical compensation characteristic.

[0075] For example, after retaining the local extreme points corresponding to the actual changes in bending stiffness, the arc length coordinates of each local extreme point relative to the base point of the bud stem on the bud stem contour segment are recorded. Local extreme points with adjacent arc length coordinates and a spacing less than a preset spacing threshold are merged into the same bending change segment, resulting in several bending change segments. The preset spacing threshold is used to merge spatially close extreme points into the same lignification change region, avoiding duplicate counting due to adjacent extreme points corresponding to the same physical location; it can be taken as 2 to 3 times the arc length interval Δs between adjacent sampling points. The larger the mechanical compensation characteristic value, the more significant the abrupt change in bending stiffness at that location in the stem, corresponding to a deeper degree of lignification in the stem.

[0076] S4. Based on the basal point of the bud stem and the key points of the leaf tip and leaf base of each leaf, construct the bud stem principal axis vector representing the direction of the bud stem and the leaf midrib vector representing the direction of the midrib of each leaf, and calculate the angle between the bud stem principal axis vector and the midrib vector of each leaf to obtain the leaf unfolding angle feature. For example, the leaf spread angle feature is used to quantitatively describe the degree of leaf spread relative to the main axis of the bud stem. Based on the bud stem base point and the leaf tip key point and leaf base key point of each leaf, two types of vectors are constructed: the vector pointing from the bud stem base point to the bud tip key point is used as the bud stem main axis vector, representing the extension direction of the bud stem; the vector pointing from the leaf base key point to the leaf tip key point of each leaf is used as the leaf midrib vector, representing the extension direction of the leaf midrib. The angle between the bud stem main axis vector and the midrib vector of each leaf is calculated. Specifically, the angle value is obtained by dividing the dot product of the two vectors by the product of the magnitudes of the two vectors and then taking the inverse cosine. The calculation formula is: θ = arccos((bud stem main axis vector · leaf midrib vector) / (|bud stem main axis vector| × |leaf midrib vector|)).

[0077] When calculating the arithmetic mean of the angles corresponding to each leaf, to ensure the comparability of the leaf unfolding angle feature values ​​in the time-series image sequence, only the average angle between the two leaves closest to the bud stem base (i.e., the first and second leaves) is uniformly taken as the leaf unfolding angle feature value. When the tea bud has only one unfolded leaf, the leaf unfolding angle feature value is the angle value corresponding to that leaf. The above convention ensures that the statistical caliber of the leaf unfolding angle feature is consistent across images of different growth stages, avoiding abrupt changes in the leaf unfolding angle feature value over time due to different leaf counts. The larger the leaf unfolding angle feature value, the higher the degree of leaf unfolding, which usually corresponds to a more mature state of tea bud growth.

[0078] S5. The mechanical compensation features and leaf spread angle features are concatenated into a multi-dimensional feature vector. The time difference label is used as the regression target. Based on the training dataset, a mapping relationship is established from the multi-dimensional feature vector to the remaining time to the optimal picking time. The method for establishing a mapping relationship from the multidimensional feature vector to the remaining time until the optimal picking time based on the training dataset includes: The mechanical compensation features and leaf unfolding angle features extracted from each historical tea bud image in the training dataset are concatenated to form a multidimensional feature vector representing the current growth state of the tea buds. At the same time, the time difference label corresponding to each frame image is numerically normalized to obtain the normalized time difference label. The mean squared error loss function is used to measure the difference between the prediction output of the initial mapping model and the normalized time difference label.

[0079] An initial mapping model based on gradient boosting regression tree is constructed, with the multidimensional feature vector as the input feature of the initial mapping model and the normalized time difference label as the output target of the initial mapping model.

[0080] Calculate the prediction residual of the current initial mapping model for each training sample. The prediction residual is the normalized time difference label minus the prediction output of the current initial mapping model. Fit a new regression tree with the multidimensional feature vector as the independent variable and the prediction residual as the target variable, and update the initial mapping model. Stop training when the mean squared error loss function no longer decreases after a preset number of consecutive rounds. Obtain the gradient boosting regression tree model as a mapping relationship between the multidimensional feature vector and the remaining time to the optimal picking time.

[0081] For example, the temporal difference label corresponding to each frame of the image is numerically normalized. The normalization method is to map the temporal difference label values ​​to the [0,1] interval, using the difference between the maximum and minimum values ​​of the temporal difference labels in the training dataset as the normalization denominator. The normalized temporal difference label is obtained by subtracting the minimum value from the temporal difference label of each frame and dividing by the normalization denominator. The formula for calculating the mean squared error loss function is: MSE=(1 / n)×Σ(y i - i )², where n is the total number of training samples, y i Let i be the normalized time difference label for the i-th sample. i This is the model's predicted output for the i-th sample.

[0082] The main hyperparameters of the gradient boosting regression tree model include the maximum depth of the decision tree, the upper limit of the number of iterations, and the learning rate. The maximum depth of the decision tree controls the complexity of a single regression tree and can be 3 to 5 layers. The upper limit of the number of iterations is set to automatically stop when the mean squared error loss function no longer decreases within a preset number of iterations; the preset number of iterations can be 50. The learning rate controls the update magnitude of each newly added regression tree to the current model and ranges from 0.05 to 0.1. The specific values ​​of these hyperparameters can be determined during the training phase using a grid search within these ranges via cross-validation, with the minimum mean squared error on the validation set as the criterion for determining the optimal hyperparameter combination.

[0083] Real-time images of the tea garden to be tested are acquired and the mechanical compensation features and leaf unfolding angle features of the tea bud samples are extracted. The harvesting time prediction result is output through the mapping relationship.

[0084] The method for acquiring real-time images of the tea garden to be tested and extracting the mechanical compensation features and leaf unfolding angle features of tea bud samples, and then outputting the harvesting time prediction result through the mapping relationship includes: The tea garden to be predicted is divided into several grid units according to a preset grid size; real-time images are acquired for each grid unit, and a preset number of tea bud samples are randomly selected from the real-time images. Mechanical compensation features and leaf unfolding angle features are extracted for each tea bud sample, and the remaining time prediction value of each tea bud sample from the optimal picking time is calculated through the mapping relationship; the arithmetic mean of the remaining time prediction values ​​of all tea bud samples in the same grid unit is calculated as the picking time prediction result and output.

[0085] For example, the tea garden to be predicted is divided into several grid units according to a preset grid size. The preset grid size can be determined based on the tea garden area, the row spacing of the tea trees, and the field of view of the camera equipment, for example, 5 meters × 5 meters as one grid unit. Real-time images are acquired for each grid unit, and a preset number of tea bud samples are randomly selected from the real-time images. It is recommended to randomly select no less than 10 tea bud samples in each grid unit to ensure the statistical stability of the arithmetic mean of the remaining time prediction values. For each tea bud sample, tea bud instance segmentation, key point localization, mechanical compensation feature extraction, and leaf unfolding angle feature extraction are performed sequentially to form a multi-dimensional feature vector, which is input into the trained gradient boosting regression tree model. The normalized prediction value output by the model is denormalized to obtain the remaining time prediction value in hours. The arithmetic mean of the remaining time prediction values ​​of all tea bud samples in the same grid unit is calculated, and this arithmetic mean is output as the harvesting time prediction result for that grid unit. If the standard deviation of the remaining time prediction value of each tea bud sample in the same grid cell exceeds the preset dispersion threshold (e.g., more than 6 hours), it indicates that the growth status of tea buds in the grid cell is unevenly distributed. It is recommended to subdivide the grid cell or increase the number of samples and then re-predict, so as to achieve refined management of the harvesting time in each area of ​​the tea garden.

[0086] Example 2: Based on the same inventive concept, such as Figure 2 As shown, this embodiment also provides a tea picking time prediction system based on tea image analysis, the system comprising: The training dataset construction and labeling module is used to collect multiple time-series images of historical sample tea buds during their complete growth cycle as the training dataset; the optimal picking frame is determined from the multiple time-series images, and each frame in the multiple time-series images is labeled with a time difference label between the optimal picking frame and the optimal picking frame.

[0087] The tea bud key point detection and localization module is used to segment tea bud instances in each frame of the training dataset and extract the tea bud foreground region; extract the skeleton of the tea bud foreground region to obtain the tea bud skeleton line; and detect and locate the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the base of each leaf on the tea bud skeleton line.

[0088] The mechanical compensation feature extraction module is used to extract the bud and stem outline segment from the outline of the tea bud foreground area based on the relative positional relationship between the bud and stem base point and the leaf base key point of each leaf, and calculate the curvature change rate on the bud and stem outline segment as a mechanical compensation feature characterizing the degree of lignification of the bud and stem.

[0089] The leaf unfolding angle feature extraction module is used to construct the bud stem principal axis vector representing the direction of the bud stem and the leaf midrib vector representing the direction of the midrib of each leaf based on the bud stem base point and the leaf tip key point and leaf base key point of each leaf, and calculate the angle between the bud stem principal axis vector and the midrib vector of each leaf to obtain the leaf unfolding angle feature.

[0090] The picking time prediction output module is used to concatenate the mechanical compensation features and leaf unfolding angle features into a multi-dimensional feature vector, take the time difference label as the regression target, and establish a mapping relationship from the multi-dimensional feature vector to the remaining time from the optimal picking time based on the training dataset; acquire real-time images of the tea garden to be tested and extract the mechanical compensation features and leaf unfolding angle features of the tea bud samples, and output the picking time prediction result through the mapping relationship.

[0091] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0092] Finally, it should be noted that although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting tea picking time based on tea leaf image analysis, characterized in that, The method includes: Collect multiple time-series images of historical tea buds throughout their complete growth cycle as a training dataset; determine the optimal picking frame from the multiple time-series images, and label each frame in the multiple time-series images with the time difference between the optimal picking frame and the optimal picking frame. Tea bud instance segmentation is performed on each frame of the training dataset to extract the tea bud foreground region; the skeleton of the tea bud foreground region is extracted to obtain the tea bud skeleton line; the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the key points of the leaf base of each leaf are detected and located on the tea bud skeleton line. Based on the relative positional relationship between the bud stem base point and the key leaf base point of each leaf, a bud stem outline segment is extracted from the outline of the tea bud foreground area, and the curvature change rate on the bud stem outline segment is calculated as a mechanical compensation feature characterizing the degree of lignification of the bud stem. Based on the basal point of the bud stem and the key points of the leaf tip and leaf base of each leaf, construct the bud stem principal axis vector representing the direction of the bud stem and the leaf midrib vector representing the direction of the midrib of each leaf. Calculate the angle between the bud stem principal axis vector and the midrib vector of each leaf to obtain the leaf unfolding angle feature. The mechanical compensation features and leaf unfolding angle features are concatenated into a multidimensional feature vector. The time difference label is used as the regression target. A mapping relationship is established from the multidimensional feature vector to the remaining time from the optimal picking time based on the training dataset. Real-time images of the tea garden to be tested are acquired and the mechanical compensation features and leaf unfolding angle features of the tea bud samples are extracted. The picking time prediction result is output through the mapping relationship.

2. The method for predicting tea picking time based on tea image analysis according to claim 1, characterized in that, The method for determining the optimal capture frame from the multiple time-series images includes: The collected historical tea buds were sorted into a time sequence by the order of collection time. The leaf unfolding angle features were extracted from each frame of the time sequence to obtain a leaf unfolding angle feature value sequence. Calculate the first-order difference between two adjacent frames in the leaf angle feature value sequence to obtain the leaf angle change rate sequence; take the frame corresponding to the position where the change rate value in the leaf angle change rate sequence first drops to less than the preset change rate threshold as the starting frame when the leaf angle tends to stabilize. Starting from the initial frame, a preset number of subsequent images are continuously taken in the time-increasing direction to form a stable window. The arithmetic mean of the leaf angle feature values ​​of all frames in the stable window is calculated as the average leaf angle during the stable period, and the standard deviation of the leaf angle feature values ​​of all frames in the stable window is calculated as the fluctuation amplitude during the stable period. When the fluctuation amplitude during the stable period is less than the preset fluctuation threshold, the frame image in the middle of the time within the stable window is marked as the candidate best capture frame; the mechanical compensation feature of the candidate best capture frame is obtained. If the mechanical compensation feature is less than the preset mechanical compensation threshold, the candidate best capture frame is determined as the best capture frame; otherwise, backtracking along the decreasing time direction to the first frame image before the stable window, the first frame image is used as the new candidate best capture frame, and the mechanical compensation feature of the new candidate best capture frame is compared with the preset mechanical compensation threshold again, until a frame image with a mechanical compensation feature less than the preset mechanical compensation threshold is found as the best capture frame.

3. The method for predicting tea picking time based on tea image analysis according to claim 1, characterized in that, The method for calculating the rate of curvature change on the bud-stem contour segment as a mechanical compensatory feature characterizing the degree of lignification of the bud-stem includes: On the outline segment of the bud stem extracted from the outline of the tea bud foreground region, a sequence of outline sampling points with equal arc length intervals is extracted along the extension direction of the bud stem; a local arc is fitted according to the position coordinates of every three adjacent sampling points in the outline sampling point sequence, and the reciprocal of the radius of the local arc is used as the curvature value of the middle sampling point to obtain a discrete curvature sequence along the bud stem axis. The discrete curvature sequence is subjected to a difference operation along the bud stem axis to obtain the curvature difference value between adjacent sampling points, and the curvature difference value is divided by the arc length interval between adjacent sampling points to obtain the curvature change rate sequence along the bud stem axis. The maximum absolute value of the rate of change of curvature is extracted from the curvature change rate sequence as a mechanical compensation feature.

4. The method for predicting tea picking time based on tea image analysis according to claim 3, characterized in that, The method for extracting a sequence of contour sampling points with equal arc length intervals along the extension direction of the bud stem on the bud stem contour segment extracted from the contour of the tea bud foreground region includes: After extracting the bud and stem outline segment from the outline of the tea bud foreground area, curve fitting is performed on the bud and stem outline segment to obtain a continuous parametric curve expression of the bud and stem outline; on the continuous parametric curve expression of the bud and stem outline, the total arc length of the bud and stem outline is calculated with the bud and stem base point as the starting point and the connection point between the bud and stem and the leaf as the ending point. The total arc length of the bud and stem outline is divided into a preset number of arc length intervals. At the endpoint of each arc length interval, the coordinate points on the continuous parameterized curve expression of the bud and stem outline are extracted to form a sequence of contour sampling points with equal arc length intervals extracted along the extension direction of the bud and stem.

5. The method for predicting tea picking time based on tea image analysis according to claim 4, characterized in that, The method for curve fitting the bud-stem contour segment to obtain a continuous parametric curve expression for the bud-stem contour includes: Extract the coordinate sequence of all contour pixels on the bud and stem contour segment, and perform sliding window averaging filtering on the coordinate sequence to eliminate jagged noise at the image segmentation boundary, so as to obtain a smoothed contour pixel coordinate sequence. The smoothed contour pixel coordinate sequence is interpolated and fitted using a cubic spline interpolation method to generate a continuous parameterized curve expression for the bud and stem contour with arc length as the independent variable. The fitting deviation at each contour pixel is calculated based on the continuous parameterized curve expression of the bud and stem contour. If the fitting deviation at any contour pixel exceeds a preset deviation threshold, the contour pixel is removed and cubic spline interpolation is performed again until the fitting deviation at all retained contour pixels does not exceed the preset deviation threshold, thus obtaining the continuous parameterized curve expression of the bud and stem contour.

6. The method for predicting tea picking time based on tea image analysis according to claim 3, characterized in that, The method for extracting the maximum absolute value of the rate of change of curvature from the curvature change rate sequence as a mechanical compensation feature includes: Extreme points are detected along the bud-stem axis of the curvature change rate sequence, and all local extreme points in the curvature change rate sequence whose absolute value of curvature change rate is greater than the absolute value of curvature change rate of adjacent sampling points are extracted. A local curvature change rate window is formed by taking a preset number of sampling points forward and backward along the bud stem axis, with each local extreme point as the center. The local standard deviation of the absolute value of the curvature change rate is calculated within the local curvature change rate window. If the local standard deviation is less than the preset standard deviation threshold, it is determined that there is a change in the actual bending stiffness at the bud stem contour position corresponding to the local extreme point due to the abrupt change in the degree of lignification. If the local standard deviation is greater than the preset standard deviation threshold, the bud and stem contour position corresponding to the local extreme point is determined to be a pseudo-bending change point and is removed. From the local extreme points corresponding to the retained true bending stiffness changes, the maximum absolute value of the rate of change of curvature is selected as the mechanical compensation feature.

7. The method for predicting tea picking time based on tea image analysis according to claim 6, characterized in that, The method of selecting the maximum absolute value of the rate of change of curvature from the local extreme points corresponding to the retained true bending stiffness changes as the mechanical compensation feature includes: Obtain all local extreme points corresponding to the actual bending stiffness changes, and record the arc length coordinates of each local extreme point corresponding to the actual bending stiffness change on the bud-stem contour segment relative to the bud-stem base point. Based on the arc length coordinates of the local extreme points corresponding to each actual bending stiffness change, the local extreme points with adjacent arc length coordinates and a spacing less than a preset spacing threshold are merged into the same bending change segment to obtain at least one bending change segment. Calculate the segment average of the absolute values ​​of the rate of curvature change at all local extreme points within each bending change segment, and use the segment average as the bending abrupt change intensity of the bending change segment; select the bending change segment with the largest bending abrupt change intensity from all bending change segments, and within the bending change segment with the largest bending abrupt change intensity, select the maximum absolute value of the rate of curvature change as the mechanical compensation characteristic.

8. The method for predicting tea picking time based on tea image analysis according to claim 1, characterized in that, The method for establishing a mapping relationship from the multidimensional feature vector to the remaining time until the optimal picking time based on the training dataset includes: The mechanical compensation features and leaf unfolding angle features extracted from each historical tea bud image in the training dataset are spliced ​​together to form a multidimensional feature vector representing the current growth state of the tea buds. At the same time, the time difference label corresponding to each frame image is numerically normalized to obtain the normalized time difference label. The mean squared error loss function is used to measure the difference between the prediction output of the initial mapping model and the normalized time difference label. An initial mapping model based on gradient boosting regression tree is constructed, with the multidimensional feature vector as the input feature of the initial mapping model and the normalized time difference label as the output target of the initial mapping model. Calculate the prediction residual of the current initial mapping model for each training sample. The prediction residual is the normalized time difference label minus the prediction output of the current initial mapping model. Fit a new regression tree with the multidimensional feature vector as the independent variable and the prediction residual as the target variable, and update the initial mapping model. Stop training when the mean squared error loss function no longer decreases after a preset number of consecutive rounds. Obtain the gradient boosting regression tree model as a mapping relationship between the multidimensional feature vector and the remaining time to the optimal picking time.

9. The method for predicting tea picking time based on tea image analysis according to claim 1, characterized in that, The method for acquiring real-time images of the tea garden to be tested and extracting the mechanical compensation features and leaf unfolding angle features of tea bud samples, and then outputting the harvesting time prediction result through the mapping relationship includes: The tea garden to be predicted is divided into several grid units according to a preset grid size; real-time images are acquired for each grid unit, and a preset number of tea bud samples are randomly selected from the real-time images. Mechanical compensation features and leaf unfolding angle features are extracted for each tea bud sample, and the remaining time prediction value of each tea bud sample from the optimal picking time is calculated through the mapping relationship; the arithmetic mean of the remaining time prediction values ​​of all tea bud samples in the same grid unit is calculated as the picking time prediction result and output.

10. A tea picking time prediction system based on tea image analysis, characterized in that, The system includes: The training dataset construction and labeling module is used to collect multiple time-series images of historical tea buds during their complete growth cycle as the training dataset; the optimal picking frame is determined from the multiple time-series images, and each frame in the multiple time-series images is labeled with a time difference label between the optimal picking frame and the optimal picking frame. The tea bud key point detection and localization module is used to segment tea bud instances in each frame of the training dataset and extract the tea bud foreground region; extract the skeleton of the tea bud foreground region to obtain the tea bud skeleton line; and detect and locate the key points of the bud tip, the base of the bud stem, and the key points of the leaf tip and the base of each leaf on the tea bud skeleton line. The mechanical compensation feature extraction module is used to extract the bud and stem outline segment from the outline of the tea bud foreground area based on the relative positional relationship between the bud and stem base point and the leaf base key point of each leaf, and calculate the curvature change rate on the bud and stem outline segment as a mechanical compensation feature characterizing the degree of lignification of the bud and stem. The leaf unfolding angle feature extraction module is used to construct the main axis vector of the bud stem representing the direction of the bud stem and the midrib vector of the leaf representing the direction of the midrib of each leaf based on the bud stem base point and the leaf tip key point and leaf base key point of each leaf, and calculate the angle between the main axis vector of the bud stem and the midrib vector of each leaf to obtain the leaf unfolding angle feature. The picking time prediction output module is used to concatenate the mechanical compensation features and leaf unfolding angle features into a multi-dimensional feature vector, take the time difference label as the regression target, and establish a mapping relationship from the multi-dimensional feature vector to the remaining time from the optimal picking time based on the training dataset; acquire real-time images of the tea garden to be tested and extract the mechanical compensation features and leaf unfolding angle features of the tea bud samples, and output the picking time prediction result through the mapping relationship.

Images