Tomato picking node analysis method and system based on image recognition

Abstract

The invention relates to the technical field of image recognition, in particular to a tomato picking node analysis method and system based on image recognition, and the method comprises the steps: obtaining first pose change data and second pose change data from a pose change video stream of tomatoes blown by airflow with preset intensity, and calculating the growth toughness characterization coefficient of the tomatoes; determining a state transition characteristic time interval of tomato stem growth on a coefficient change curve of the growth toughness characterization coefficient; the method comprises the following steps: determining a frame selection region in a tomato identification image, and obtaining RGB color dominant values of region pixel points in the frame selection region; and finally, based on a comparison result of the RGB color dominant values corresponding to the frame selection areas in the tomato identification images of the adjacent time nodes, determining whether the tomatoes have picking conditions or not. According to the method, the lignification growth degree of the tomato stems is represented through multi-dimensional data, and the requirements for precise tomato picking and quality and efficiency improvement in modern agriculture are met.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for analyzing tomato picking nodes based on image recognition. Background Technology

[0002] In current tomato harvesting operations, the determination of the harvesting point largely relies on manual experience. The timing of harvesting is determined by visually observing the appearance of the tomato body, stem, and fruit maturity. This method is greatly affected by subjective factors, and the judgment criteria are not uniform. It is easy to cause problems such as harvesting too early, resulting in insufficient fruit maturity, or harvesting too late, resulting in excessive lignification of the stem, leading to stem breakage and fruit drop during harvesting. This significantly reduces the efficiency of tomato harvesting and the quality of the fruit.

[0003] To meet the mechanized harvesting needs of large-scale tomato cultivation, existing mechanized harvesting node determination technologies mostly rely on a single fruit color feature to identify maturity, without combining the core harvesting adaptation feature of the growth resilience changes of the tomato stem and the tomato body. The determination method based on a single color feature cannot accurately match the actual growth state of the tomato stem, which easily leads to a mismatch between the harvesting time and the plant's growth state.

[0004] For example, Chinese invention patent CN114155526A discloses a method, device, equipment, and product for predicting tomato fruit growth, relating to the field of agricultural technology. The method includes the following steps: acquiring a canopy spectral image of the tomato canopy, and extracting leaf spectral images containing only leaves and binary images of fruit morphology containing only fruits; inputting the leaf spectral images containing only leaves into a leaf neural network model to obtain leaf parameters; performing corner detection analysis on the binary images of fruit morphology containing only fruits to obtain the number of tomato fruits; obtaining the phloem sugar concentration under different planting conditions based on the canopy spectral image, leaf parameters, and the number of tomato fruits; and inputting the phloem sugar concentration under different planting conditions into a tomato fruit growth model to obtain the dynamic process of tomato fruit fresh weight and dry weight growth. This improves the prediction accuracy of tomato fruit growth.

[0005] Current technologies do not consider that lignification of tomato stems is a key factor affecting harvesting timing. Existing technologies cannot characterize the degree of lignification of tomato stems through multi-dimensional data, and therefore cannot meet the needs of modern agriculture for precise tomato harvesting, quality improvement and efficiency enhancement. Summary of the Invention

[0006] To address this, the present invention provides a method and system for analyzing tomato harvesting nodes based on image recognition, which overcomes the problem that existing technologies cannot characterize the degree of lignification growth of tomato stems through multi-dimensional data, and thus cannot meet the needs of precise tomato harvesting, quality improvement and efficiency enhancement in modern agriculture.

[0007] To achieve the above objectives, this invention provides a tomato harvesting node analysis method based on image recognition, comprising:

[0008] The video stream shows the pose changes of a tomato at several time points as it is blown by an airflow of preset intensity. Based on the first pose change data and the second pose change data of the tomato in the video stream, the growth toughness characterization coefficient of the tomato at each time point is calculated.

[0009] Plot the coefficient change curve of the growth resilience characterization coefficient over time, and determine the time interval of the tomato stem growth state transition characteristics based on the curve information of the data points on the coefficient change curve.

[0010] Several frames of tomato recognition images are selected from the pose change video stream within the state transition feature time interval. A bounding box region is determined in the tomato recognition images, and the RGB color dominance values ​​of the region pixels are obtained in the bounding box region.

[0011] The selected area includes several sub-regions of the tomato recognition image;

[0012] By comparing the RGB color dominance values ​​of the selected areas in the tomato recognition images at adjacent time points in chronological order, the method determines whether the tomatoes are ready for harvesting based on the comparison results of the RGB color dominance values.

[0013] Furthermore, the process of obtaining the first pose change data of the tomato in the pose change video stream includes:

[0014] Establish a spatial coordinate system in which the X and Y axes are parallel to the horizontal ground and the Z axis is perpendicular to the horizontal ground;

[0015] Obtain the centroid coordinates of a single tomato's outline in the spatial coordinate system, and determine the difference between the maximum and minimum Z-axis coordinates of the centroid coordinates under a preset intensity of airflow as the first pose change data.

[0016] Furthermore, the process of acquiring the second pose change data of the tomato in the pose change video stream includes:

[0017] Obtain the centroid coordinates of a single tomato's outline in the spatial coordinate system, and determine the maximum offset distance of the centroid coordinates on the XY plane under a preset intensity airflow as the second pose change data.

[0018] Furthermore, the process of calculating the growth resilience characterization coefficient of tomatoes at each time point includes:

[0019] The first pose change data and the second pose change data at each time point are weighted and summed, and the result of the weighted summation is determined as the growth toughness characterization coefficient.

[0020] Furthermore, the process of determining the time intervals characteristic of the state transition in tomato stem growth includes:

[0021] Determine the slope value of the data points corresponding to each time node on the coefficient change curve;

[0022] Calculate the slope difference between the slope of the data point corresponding to the later time node and the slope of the data point corresponding to the earlier time node in two adjacent data points in the time series.

[0023] If the slope difference is greater than the preset slope difference reference value, then the time interval determined by the time nodes corresponding to the two adjacent data points is determined as the time interval of the state transformation characteristics of tomato stem growth.

[0024] Furthermore, the process of determining the selected area includes:

[0025] Contour recognition is performed on the tomato recognition image to extract the contour of the tomato body and the contour of the stem that connects to the contour of the tomato body.

[0026] Obtain the contour connection point between the tomato body contour and the stem contour;

[0027] The line that passes through the intersection point of the outline and is parallel to the horizontal ground is determined as the region segmentation baseline. The stem outline sub-region is determined above the region segmentation baseline, and the tomato body outline sub-region is determined below the region segmentation baseline.

[0028] The stem outline sub-region is the same size as the tomato body outline sub-region.

[0029] Furthermore, the process of obtaining the RGB color luminance values ​​of the pixels in the selected area includes:

[0030] Select several pixels within the stem outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the stem outline sub-region as the first RGB color luminance value.

[0031] Select several pixels within the tomato body outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the tomato body outline sub-region as the second RGB color luminance value.

[0032] Furthermore, the process of comparing the RGB color dominance values ​​of the selected areas in the tomato recognition images at adjacent time points includes:

[0033] Obtain the first RGB color dominance value and the second RGB color dominance value at the i-th time node, and the first RGB color dominance value and the second RGB color dominance value at the (i+1)-th time node, respectively;

[0034] Calculate the first difference between the first RGB color dominance value at the (i+1)th time node and the first RGB color dominance value at the ith time node, and calculate the second difference between the second RGB color dominance value at the (i+1)th time node and the second RGB color dominance value at the ith time node.

[0035] Furthermore, the process of determining whether tomatoes are ready for harvest includes:

[0036] If the first difference or the second difference is negative, and the first difference or the second difference is less than or equal to a preset difference threshold, then the tomatoes are determined to be ready for picking.

[0037] If neither the first difference nor the second difference is negative, or if either the first difference or the second difference is greater than or equal to a preset difference threshold, then the tomatoes are determined not to be ready for picking.

[0038] Furthermore, the present invention also provides a tomato harvesting node analysis system based on image recognition, comprising:

[0039] The airflow jet module is used to spray airflow onto tomatoes at a preset intensity for a preset duration.

[0040] A video stream extraction module, which is connected to an airflow jet module, includes a video stream acquisition unit and an extraction unit. The video stream acquisition unit is used to acquire a video stream showing the pose changes of a tomato being blown by an airflow of preset intensity at several time points.

[0041] The extraction unit is connected to the video stream acquisition unit to obtain the first pose change data and the second pose change data of the tomato, so as to calculate the growth toughness characterization coefficient of the tomato at each time node.

[0042] An image data conversion module, which is connected to the video stream extraction module, is used to plot the coefficient change curve of the growth toughness characterization coefficient, and to determine the time interval of the state transition characteristics of tomato stem growth based on the curve information of data points.

[0043] The image processing module, which is connected to the image data conversion module, is used to select several frames of tomato recognition images and obtain the RGB color display values ​​of the pixels in the selected area of ​​the tomato recognition image.

[0044] The identification and analysis module, which is connected to the image processing module, is used to determine whether tomatoes are ready for harvesting based on the comparison results of the RGB color saturation values ​​corresponding to the selected areas in the tomato identification images at adjacent time points.

[0045] The beneficial effects of the technical solution presented in this application include: calculating the growth resilience characterization coefficient of tomatoes by acquiring first and second pose change data from a video stream showing the pose changes of tomatoes being blown by a preset intensity airflow; determining the time interval of the state transition characteristics of tomato stem growth on the coefficient change curve of the growth resilience characterization coefficient; selecting several frames of tomato recognition images from the pose change video stream, defining a bounding box region in the tomato recognition images, and acquiring the RGB color dominance values ​​of the pixels in the bounding box region; finally, determining whether the tomatoes are ready for harvesting based on the comparison results of the RGB color dominance values ​​corresponding to the bounding box regions in tomato recognition images at adjacent time nodes. Furthermore, by characterizing the lignification growth degree of the tomato stem through multi-dimensional data, the needs of precise tomato harvesting and improved quality and efficiency in modern agriculture are met.

[0046] Furthermore, this invention utilizes the physical phenomenon that during the growth and ripening of tomatoes, the part where the tomato stem connects to the tomato body gradually develops. This process causes the cell wall structure of the stem to loosen and the fiber strength to continuously decrease, which in turn leads to a significant reduction in the overall toughness of the stem and a weakening of its ability to pull and restrain the fruit. By uniformly applying airflow of a preset intensity to the tomato, the invention captures the more obvious spatial swaying motion of the fruit under the disturbance of the airflow.

[0047] Furthermore, as tomatoes mature, the regulatory effects of hormones such as ethylene intensify, leading to a rapid decrease in stem toughness. At this point, the absolute value of the slope of the coefficient change curve increases significantly. By comparing this slope difference with a preset reference value, it can be determined that stem growth has entered a period of accelerated toughness reduction. The time interval corresponding to these two adjacent data points is then defined as the state transition characteristic time interval, effectively avoiding the waste of computational resources caused by an excessively large monitoring time range.

[0048] Furthermore, by extracting the G channel values ​​of two equally sized sub-regions, the present invention makes the obtained color dominance values ​​more representative, and can truly and objectively reflect the degree of stem lignification and abscission development.

[0049] Furthermore, when the first or second difference in this invention is negative, it indicates a decrease in the G channel value of the corresponding region, signifying a reduction in chlorophyll in that region and an accelerated process of stem lignification and abscission layer development. This negative difference is compared with a preset difference threshold. When the difference is less than or equal to the threshold, it indicates that the decrease in the green channel has reached a level that can stably characterize a significant reduction in the toughness of the fruit stalk, reaching a level suitable for harvesting. The tomato is then determined to be ready for harvesting. This invention characterizes the degree of lignification growth of the tomato stem through multi-dimensional data, meeting the needs of precise tomato harvesting, quality improvement, and efficiency enhancement in modern agriculture. Attached Figure Description

[0050] Figure 1 This is a step diagram of the tomato picking node analysis method based on image recognition in an embodiment of the present invention;

[0051] Figure 2 A flowchart illustrating the steps for determining the time interval of state transition characteristics in an embodiment of the present invention;

[0052] Figure 3 A flowchart illustrating the steps for determining the selected area in an embodiment of the present invention;

[0053] Figure 4 This is a schematic diagram of the stem outline sub-region and the tomato body outline sub-region in an embodiment of the present invention;

[0054] Figure 5 This is a system block diagram of the tomato harvesting node analysis system based on image recognition, according to an embodiment of the present invention.

[0055] In the diagram: 1-Tomato body outline, 2-Stem outline, 3-Outline junction, 4-Region segmentation baseline, 5-Stem outline sub-region, 6-Tomato body outline sub-region. Detailed Implementation

[0056] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0057] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] It should be noted that in the description of this invention, the terms "upper," "lower," "inner," "outer," etc., which indicate the direction or positional relationship, are based on the direction or positional relationship shown in the accompanying drawings. This is only for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0059] It should be understood that although the present invention may use terms such as "first," "second," etc., to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present invention, first information may also be referred to as second information, and similarly, second information may also be referred to as first information.

[0060] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0061] Please see Figure 1 The diagram illustrates the steps of the tomato harvesting node analysis method based on image recognition according to an embodiment of the present invention. The tomato harvesting node analysis method based on image recognition of the present invention includes:

[0062] Step S100: Obtain a video stream of the tomato's pose change when it is blown by a preset intensity airflow at several time points, and calculate the growth toughness characterization coefficient of the tomato at each time point based on the first pose change data and the second pose change data of the tomato in the pose change video stream.

[0063] In this invention, the preset airflow intensity is 0.5 m / s, the airflow nozzle is 30 cm away from the tomato fruit, the duration of a single airflow jet is 5 s, and it acts on the centroid of the tomato fruit's outline. The duration of a pose change video stream is from the start of airflow jetting to the stop of airflow jetting.

[0064] In this invention, the interval between adjacent time points is 8 hours.

[0065] It is understandable that during the growth and ripening of tomatoes, the area where the tomato stem connects to the tomato body gradually develops. This process causes the cell wall structure of the stem to loosen and the fiber strength to continue to decline, which in turn leads to a significant reduction in the overall toughness of the stem and a weakening of its ability to pull and restrain the fruit. When a preset intensity of airflow is applied evenly to the tomato, the stem, due to its insufficient toughness, cannot effectively counteract the force brought by the airflow, and the fruit will produce more obvious spatial swaying under the disturbance of the airflow.

[0066] Step S200: Plot the coefficient change curve of the growth resilience characterization coefficient over time, and determine the time interval of the tomato stem growth state transition characteristics based on the curve information of the data points on the coefficient change curve.

[0067] In this invention, a coefficient variation curve can be plotted using linear interpolation. The horizontal axis of the coefficient variation curve represents time, and the vertical axis represents the growth toughness characterization coefficient. Fitting the curve using linear interpolation is an existing technique and will not be elaborated here.

[0068] Step S300: Select several frames of tomato recognition images from the pose change video stream within the state transition feature time interval, determine a bounding box region in the tomato recognition images, and obtain the RGB color dominance value of the region pixels in the bounding box region.

[0069] The selected area includes several sub-regions of the tomato recognition image;

[0070] In this invention, five tomato recognition images are selected at equal time intervals within the state transition feature time interval, and the time interval between adjacent tomato recognition images is 1 second.

[0071] Step S400: Compare the RGB color dominance values ​​of the selected areas in the tomato recognition images of adjacent time nodes in chronological order, and determine whether the tomatoes are ready for picking based on the comparison results of the RGB color dominance values.

[0072] Specifically, the process of obtaining the first pose change data of the tomato in the pose change video stream includes:

[0073] Establish a spatial coordinate system in which the X and Y axes are parallel to the horizontal ground and the Z axis is perpendicular to the horizontal ground;

[0074] Obtain the centroid coordinates of a single tomato's outline in the spatial coordinate system, and determine the difference between the maximum and minimum Z-axis coordinates of the centroid coordinates under a preset intensity of airflow as the first pose change data.

[0075] In this invention, the unit of the first pose change data is mm.

[0076] This invention does not specifically limit the method for determining the centroid of a single tomato. It can be achieved by performing noise reduction preprocessing on a single frame image in a pose change video stream, using edge detection or threshold segmentation algorithms to identify and extract the complete closed contour of a single tomato fruit, and determining the geometric center of the complete closed contour as the centroid of the tomato. The edge detection or threshold segmentation algorithms are commonly used algorithms in image recognition processing and are well-known in the field of image processing, so they will not be elaborated here.

[0077] It is understandable that when a preset airflow acts on a tomato, the fruit will be driven by the airflow to swing naturally in the vertical direction. At this time, the maximum and minimum values ​​of the centroid Z-axis coordinate during the swing process are extracted, and the difference between the two is used as the first posture change data. The first posture change data quantifies the vertical swing amplitude of the tomato under airflow disturbance. The vertical swing amplitude is related to the toughness of the tomato stem. The higher the toughness of the stem, the stronger the pulling and fixing effect on the fruit, the smaller the vertical swing amplitude under the airflow, and the smaller the corresponding Z-axis coordinate difference. Conversely, the lower the toughness, the larger the vertical swing amplitude, and the larger the Z-axis coordinate difference.

[0078] Specifically, the process of obtaining the second pose change data of the tomato in the pose change video stream includes:

[0079] Obtain the centroid coordinates of a single tomato outline in the spatial coordinate system, and determine the maximum offset distance of the centroid coordinates on the XY plane under the blowing of a preset intensity airflow as the second pose change data.

[0080] In this invention, the second pose change data is the maximum distance between the centroid coordinates of the tomato outline and the initial static position coordinates on the XY plane. The unit of the second pose change data is mm.

[0081] It is understandable that when a preset intensity of airflow acts uniformly on a tomato, the stem will cause the fruit to sway horizontally. At this time, the XY plane coordinates of the tomato's outline centroid are continuously collected and tracked throughout the entire process of airflow. The offset distance of the centroid relative to the initial position on the horizontal plane is calculated, and the maximum offset distance is extracted as the second pose change data. The second pose change data characterizes the horizontal swaying amplitude of the tomato under airflow disturbance. This phenomenon is related to the toughness of the tomato stem. The higher the toughness of the stem, the stronger its resistance to bending and swaying, and the more obvious its horizontal pulling constraint effect on the fruit. The horizontal offset distance of the fruit under airflow is smaller. Conversely, the lower the toughness of the stem and the higher the degree of lignification, the weaker its bending constraint ability, and the greater the horizontal offset distance of the fruit affected by airflow.

[0082] Specifically, the process of calculating the growth resilience characterization coefficient of tomatoes at each time point includes:

[0083] The first pose change data k1 and the second pose change data k2 at each time point are weighted and summed, and the result of the weighted summation is determined as the growth toughness characterization coefficient K.

[0084] In this invention, the growth resilience characterization coefficient K = α × k1 + β × k2, where α is the weight value of the first pose change data k1 and β is the weight value of the second pose change data k2, and α + β = 1. In this invention, since the first pose change data k1 characterizes the vertical swing difference of the tomato fruit under the influence of a preset intensity airflow, its deformation amplitude is easily affected by the tomato's own gravity. Its response sensitivity to changes in stem resilience is relatively lower than that of horizontal swing characteristics. In order to weaken gravity interference and improve the accuracy of the growth resilience characterization coefficient in characterizing the true resilience state of the stem, a relatively small weight value is assigned to the first pose change data k1 and a relatively large weight value is assigned to the second pose change data k2. Optionally, α = 0.4 and β = 0.6.

[0085] Understandably, during the actual growth of tomatoes, high stem toughness indicates a tight cell wall structure, high fiber strength, and high bending stiffness. Under the influence of airflow, the stem's resistance to deformation is strong, resulting in only minor elastic deformation. This leads to smaller first and second posture data for the fruit. As the stem matures, abscission forms, causing cell wall relaxation and decreased fiber strength, significantly reducing the stem's bending stiffness. The same airflow force will induce a larger swaying amplitude. The weighted summation yields a growth toughness characterization coefficient that quantitatively represents the stem's toughness.

[0086] Specifically, please refer to Figure 2 As shown, this is a flowchart illustrating the steps of determining the characteristic time interval of state transition in an embodiment of the present invention. The process of determining the characteristic time interval of state transition in tomato stem growth includes:

[0087] Step S201: Determine the slope value of the data points corresponding to each time node on the coefficient change curve;

[0088] In this invention, the growth toughness characterization coefficient is used as the vertical axis and the time node as the horizontal axis. For any target data point corresponding to a target time node on the coefficient change curve, a three-point data group is formed by selecting the target data point and its adjacent preceding and following data points in time sequence. The difference in growth toughness characterization coefficient between the adjacent following data point and the preceding data point is used as the numerator, and the time difference between the corresponding two time nodes is used as the denominator. The ratio of the numerator to the denominator is determined as the slope value of the target data point. For the boundary data points at the first and last positions in time sequence on the coefficient change curve, the corresponding slope value is calculated by forward difference or backward difference between adjacent two points, thereby obtaining the slope value of the data points corresponding to all time nodes.

[0089] Step S202: Calculate the slope difference between the slope value of the data point corresponding to the later time node and the slope value of the data point corresponding to the previous time node in two adjacent data points in the time series.

[0090] Step S203: If the slope difference is greater than the preset slope difference reference value, then the time interval determined by the time nodes corresponding to the two adjacent data points is determined as the time interval of the state change characteristics of tomato stem growth.

[0091] In this invention, tomato plants of the same variety as the one being monitored are selected as test samples. During the normal growth cycle from the color-changing stage to the full maturity stage, multiple sets of test samples are collected and the slope values ​​of each data point are calculated, according to the same preset airflow intensity, time interval, position data extraction method, and growth toughness characterization coefficient calculation method as this invention. The slope difference between adjacent data points is then obtained. At the same time, the corresponding growth stages of tomato stems, such as the beginning of obvious lignification, the initial formation of abscission layer, and the significant change in toughness, are marked by artificial observation. The slope difference value range corresponding to this stage is extracted. The slope difference values ​​of the mutation stages obtained from multiple tests are averaged and the calculated results are finally determined as the slope difference reference value. Optionally, the slope difference reference value is 0.08.

[0092] After continuously acquiring the growth resilience characterization coefficients at various time points and plotting them as curves, the slope of the curves intuitively reflects the rate of decrease in stem resilience. The larger the absolute value of the slope, the faster the stem resilience decreases per unit time, and the faster the fruit matures and develops. Because tomato stem growth exhibits distinct stages, in the early stages of growth, fiber strength and resilience remain stable, and the slope difference between adjacent nodes is relatively small. However, as the tomato enters the ripening stage, the regulatory effect of hormones such as ethylene intensifies, and stem resilience decreases more rapidly. At this time, the absolute value of the slope of the coefficient change curve increases significantly. By comparing this slope difference with a preset slope difference reference value, it is determined that stem growth has entered a period of accelerated resilience reduction. The time interval corresponding to these two adjacent data points is then defined as the state transition characteristic time interval, effectively avoiding the waste of computational resources caused by an excessively large monitoring time range.

[0093] Specifically, please refer to Figure 3 as well as Figure 4 As shown, Figure 3 This is a flowchart illustrating the steps for determining the selected area in an embodiment of the present invention. Figure 4 This is a schematic diagram of the stem outline sub-region and the tomato body outline sub-region in an embodiment of the present invention. The process of determining the selected area includes:

[0094] Step S301: Perform contour recognition on the tomato recognition image to extract the tomato body contour 1 and the stem contour 2 connected to the tomato body contour.

[0095] Step S302: Obtain the outline connection point 3 of the tomato body outline 1 and the stem outline 2;

[0096] In this invention, the connection point between the tomato body contour and the stem contour is determined by the contour topology relationship, which is a well-known aspect of existing image contour analysis technology and will not be elaborated here.

[0097] Step S303: The line that passes through the intersection point of the outline and is parallel to the horizontal ground is determined as the region segmentation baseline 4. The stem outline sub-region 5 is determined above the region segmentation baseline, and the tomato body outline sub-region 6 is determined below the region segmentation baseline.

[0098] Among them, the stem outline sub-region 5 and the tomato body outline sub-region 6 have the same size.

[0099] Optionally, the stem outline sub-region and the tomato body outline sub-region are rectangular regions with the same pixel area and a size of 50×50 pixels.

[0100] Understandably, contour analysis is used to find the intersection point between the tomato body contour and the stem contour. This point is the connection point between the stem and the fruit body, and it is also the core location for determining the development of abscission layer and the degree of lignification. In order to achieve symmetrical and standardized extraction of color features in the connection area between the stem and the fruit, the stem contour sub-region and the tomato body contour sub-region are delineated separately. Sub-regions of equal size can ensure that the number and range of pixel sampling are consistent, thereby focusing on the color change contrast between stem lignification and fruit maturity in the connection area. The upper stem contour sub-region captures the lignification changes at the stem end, and the lower tomato body contour sub-region captures the maturity changes at the fruit end.

[0101] Specifically, the process of obtaining the RGB color luminance values ​​of the pixels in the selected area includes:

[0102] Select several pixels within the stem outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the stem outline sub-region as the first RGB color luminance value.

[0103] Select several pixels within the tomato body outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the tomato body outline sub-region as the second RGB color luminance value.

[0104] In this invention, because the tomato stem undergoes abscission formation and lignification during the ripening process, the chlorophyll content continuously decreases, the green characteristics weaken accordingly, and the G channel value shows a stable downward trend. However, the G channel change pattern of the tomato fruit body is significantly different from that of the stem. By extracting the G channel values ​​of two equal-sized sub-regions, the obtained color dominance values ​​are more representative and can truly and objectively reflect the degree of lignification and abscission development of the stem.

[0105] Specifically, the process of comparing the RGB color dominance values ​​of the selected areas in the tomato recognition images at adjacent time points includes: Obtain the first RGB color dominance value and the second RGB color dominance value at the i-th time node, and the first RGB color dominance value and the second RGB color dominance value at the (i+1)-th time node, respectively; Calculate the first difference between the first RGB color dominance value at the (i+1)th time node and the first RGB color dominance value at the ith time node, and calculate the second difference between the second RGB color dominance value at the (i+1)th time node and the second RGB color dominance value at the ith time node.

[0106] In this invention, the first difference is equal to the first RGB color dominance value at the (i+1)th time node minus the first RGB color dominance value at the ith time node; the second difference is equal to the second RGB color dominance value at the (i+1)th time node minus the second RGB color dominance value at the ith time node.

[0107] Specifically, the process of determining whether tomatoes are ready for harvest includes:

[0108] If the first difference or the second difference is negative, and the first difference or the second difference is less than or equal to a preset difference threshold, then the tomatoes are determined to be ready for picking.

[0109] If neither the first difference nor the second difference is negative, or if either the first difference or the second difference is greater than or equal to a preset difference threshold, then the tomatoes are determined not to be ready for picking.

[0110] Tomato plants of the same variety as the monitored object are pre-selected as test samples. During the normal growth cycle of tomatoes from color change to full maturity, multiple sets of test samples are collected at different maturity stages using the same image acquisition conditions, bounding box division method, RGB color dominance value extraction method, and difference calculation rules as in this technical solution. Simultaneously, key stages are manually observed and marked, indicating significant lignification of the tomato stem, sufficient abscission layer development, and suitability for harvesting. The difference data corresponding to these stages are extracted, and the average of the effective differences obtained from multiple tests is used as the preset difference threshold. In this invention, the preset difference threshold can be -5.

[0111] Understandably, the first difference reflects the change in the G-channel value of the stem outline sub-region over time, and the second difference reflects the change in the G-channel value of the tomato body outline sub-region over time. When either the first or second difference is negative, it indicates a decrease in the G-channel value of the corresponding region, signifying a reduction in chlorophyll in that region and an accelerated process of stem lignification and abscission layer development. This negative difference is compared with a preset difference threshold. When the difference is less than or equal to the threshold, it indicates that the decrease in the green channel has reached a level that can stably represent a significant reduction in the toughness of the fruit stalk, reaching a level suitable for harvesting. The tomato is then determined to be ready for harvesting. This invention characterizes the degree of lignification growth of the tomato stem through multi-dimensional data, meeting the needs of precise tomato harvesting, quality improvement, and efficiency enhancement in modern agriculture.

[0112] Specifically, please refer to Figure 5 The diagram shown is a system block diagram of a tomato harvesting node analysis system based on image recognition, according to an embodiment of the present invention. The present invention also provides a tomato harvesting node analysis system based on image recognition, comprising:

[0113] The airflow jet module is used to spray airflow onto tomatoes at a preset intensity for a preset duration.

[0114] The present invention does not limit the specific structure of the airflow injection module. It can be composed of an air pump, a nozzle and a control valve, as long as it can output a constant airflow with a preset intensity and a preset duration. Further details are omitted here.

[0115] A video stream extraction module, which is connected to an airflow jet module, includes a video stream acquisition unit and an extraction unit. The video stream acquisition unit is used to acquire a video stream showing the pose changes of a tomato being blown by an airflow of preset intensity at several time points.

[0116] This invention does not limit the specific structure of the video stream extraction module. The video stream acquisition unit can be implemented using image acquisition devices such as industrial cameras or high-definition cameras. The extraction unit can be constructed using logic components, such as field-programmable logic components, microprocessors, or processors used in computers, to realize image contour recognition, centroid extraction, and pose data calculation. Further details are omitted here.

[0117] The extraction unit is connected to the video stream acquisition unit to obtain the first pose change data and the second pose change data of the tomato, so as to calculate the growth toughness characterization coefficient of the tomato at each time node.

[0118] An image data conversion module, which is connected to the video stream extraction module, is used to plot the coefficient change curve of the growth toughness characterization coefficient, and to determine the time interval of the state transition characteristics of tomato stem growth based on the curve information of data points.

[0119] The present invention does not limit the specific structure of the image data conversion module. It can be a microprocessor used to realize curve fitting, slope calculation and state transition interval determination, which will not be described in detail here.

[0120] The image processing module, which is connected to the image data conversion module, is used to select several frames of tomato recognition images and obtain the RGB color display values ​​of the pixels in the selected area of ​​the tomato recognition image.

[0121] The present invention does not limit the specific structure of the image processing module, which can be an image processing CPU used to implement image segmentation and region selection, and will not be described in detail here.

[0122] The identification and analysis module, which is connected to the image processing module, is used to determine whether tomatoes are ready for harvesting based on the comparison results of the RGB color saturation values ​​corresponding to the selected areas in the tomato identification images at adjacent time points.

[0123] The present invention does not limit the specific structure of the identification and analysis module. It can be a field-programmable logic unit used to realize difference calculation and picking condition determination, which will not be elaborated here.

[0124] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for analyzing tomato picking nodes based on image recognition, characterized in that, include: The video stream shows the pose changes of a tomato at several time points as it is blown by an airflow of preset intensity. Based on the first pose change data and the second pose change data of the tomato in the video stream, the growth toughness characterization coefficient of the tomato at each time point is calculated. Plot the coefficient change curve of the growth resilience characterization coefficient over time, and determine the time interval of the tomato stem growth state transition characteristics based on the curve information of the data points on the coefficient change curve. Several frames of tomato recognition images are selected from the pose change video stream within the state transition feature time interval. A bounding box region is determined in the tomato recognition images, and the RGB color dominance values ​​of the region pixels are obtained in the bounding box region. The selected area includes several sub-regions of the tomato recognition image; By comparing the RGB color dominance values ​​of the selected areas in the tomato recognition images at adjacent time points in chronological order, the method determines whether the tomatoes are ready for harvesting based on the comparison results of the RGB color dominance values.

2. The tomato harvesting node analysis method based on image recognition according to claim 1, characterized in that, The process of obtaining the first pose change data of the tomato in the pose change video stream includes: Establish a spatial coordinate system in which the X and Y axes are parallel to the horizontal ground and the Z axis is perpendicular to the horizontal ground; Obtain the centroid coordinates of a single tomato's outline in the spatial coordinate system, and determine the difference between the maximum and minimum Z-axis coordinates of the centroid coordinates under a preset intensity of airflow as the first pose change data.

3. The tomato harvesting node analysis method based on image recognition according to claim 2, characterized in that, The process of obtaining the second pose change data of the tomato in the pose change video stream includes: Obtain the centroid coordinates of a single tomato outline in the spatial coordinate system, and determine the maximum offset distance of the centroid coordinates on the XY plane under the blowing of a preset intensity airflow as the second pose change data.

4. The tomato harvesting node analysis method based on image recognition according to claim 3, characterized in that, The process of calculating the growth resilience characterization coefficient of tomatoes at each time point includes: The first pose change data and the second pose change data at each time point are weighted and summed, and the result of the weighted summation is determined as the growth toughness characterization coefficient.

5. The tomato harvesting node analysis method based on image recognition according to claim 4, characterized in that, The process of determining the time intervals characteristic of the state transition in tomato stem growth includes: Determine the slope value of the data points corresponding to each time node on the coefficient change curve; Calculate the slope difference between the slope of the data point corresponding to the later time node and the slope of the data point corresponding to the earlier time node in two adjacent data points in the time series. If the slope difference is greater than the preset slope difference reference value, then the time interval determined by the time nodes corresponding to the two adjacent data points is determined as the time interval of the tomato stem growth state transition characteristics.

6. The tomato harvesting node analysis method based on image recognition according to claim 1, characterized in that, The process of determining the selection area includes: Contour recognition is performed on the tomato recognition image to extract the contour of the tomato body and the contour of the stem that connects to the contour of the tomato body. Obtain the contour connection point between the tomato body contour and the stem contour; The line that passes through the intersection point of the outline and is parallel to the horizontal ground is determined as the region segmentation baseline. The stem outline sub-region is determined above the region segmentation baseline, and the tomato body outline sub-region is determined below the region segmentation baseline. The stem outline sub-region is the same size as the tomato body outline sub-region.

7. The tomato harvesting node analysis method based on image recognition according to claim 6, characterized in that, The process of obtaining the RGB color dominance values ​​of pixels in the selected area includes: Select several pixels within the stem outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the stem outline sub-region as the first RGB color luminance value. Select several pixels within the tomato body outline sub-region, extract the G channel values ​​of each pixel, and determine the average G channel values ​​of the tomato body outline sub-region as the second RGB color luminance value.

8. The tomato harvesting node analysis method based on image recognition according to claim 7, characterized in that, The process of comparing the RGB color dominance values ​​of the selected areas in the tomato recognition images at adjacent time points includes: Obtain the first RGB color dominance value and the second RGB color dominance value at the i-th time node, and the first RGB color dominance value and the second RGB color dominance value at the (i+1)-th time node, respectively; Calculate the first difference between the first RGB color dominance value at the (i+1)th time node and the first RGB color dominance value at the ith time node, and calculate the second difference between the second RGB color dominance value at the (i+1)th time node and the second RGB color dominance value at the ith time node.

9. The tomato harvesting node analysis method based on image recognition according to claim 8, characterized in that, The process of determining whether tomatoes are ready for harvest includes: If the first difference or the second difference is negative, and the first difference or the second difference is less than or equal to a preset difference threshold, then the tomatoes are determined to be ready for picking. If neither the first difference nor the second difference is negative, or if either the first difference or the second difference is greater than or equal to a preset difference threshold, then the tomatoes are determined not to be ready for picking.

10. A tomato harvesting node analysis system based on image recognition, used to execute the tomato harvesting node analysis method based on image recognition as described in any one of claims 1-9, characterized in that, include: The airflow jet module is used to spray airflow onto tomatoes at a preset intensity for a preset duration. A video stream extraction module, which is connected to an airflow jet module, includes a video stream acquisition unit and an extraction unit. The video stream acquisition unit is used to acquire a video stream showing the pose changes of a tomato being blown by an airflow of preset intensity at several time points. The extraction unit is connected to the video stream acquisition unit to obtain the first pose change data and the second pose change data of the tomato, so as to calculate the growth toughness characterization coefficient of the tomato at each time node. An image data conversion module, which is connected to the video stream extraction module, is used to plot the coefficient change curve of the growth toughness characterization coefficient, and to determine the time interval of the state transition characteristics of tomato stem growth based on the curve information of data points. The image processing module, which is connected to the image data conversion module, is used to select several frames of tomato recognition images and obtain the RGB color display values ​​of the pixels in the selected area of ​​the tomato recognition image. The identification and analysis module, which is connected to the image processing module, is used to determine whether tomatoes are ready for harvesting based on the comparison results of the RGB color saturation values ​​corresponding to the selected areas in the tomato identification images at adjacent time points.

Images