Method, system and device for judging and classifying target maturity of apple picking robot

Abstract

The invention provides a target maturity judgment and classification method, system and device for an apple picking robot, and relates to the technical field of apple picking. Reflection spectrum parameters are collected through near infrared spectrum, correlation analysis is carried out on the reflection spectrum parameters, the influence of environmental factors is reduced, and the accuracy of apple picking is improved. According to the method, an apple sugar content index used for reflecting the sugar content of a target apple is generated, meanwhile, hardness data of the apple is collected and analyzed, an apple hardness index used for reflecting the hardness of the target apple is generated, and further, the deviation degree of an actual gray scale reference gray scale of image recognition is analyzed; the method comprises the following steps: generating a hue ripeness dynamic difference value used for reflecting the deviation degree of an actual gray scale reference gray scale when an apple picking robot performs image recognition on an apple, analyzing by integrating an apple sugar content index, an apple hardness index and the hue ripeness dynamic difference value, and finally outputting an apple ripeness evaluation index so as to judge whether the apple can be picked or not.

Description

Technical Field

[0001] This invention relates to the field of apple harvesting technology, specifically to a method, system, and device for determining and classifying the ripeness of apples for harvesting robots. Background Technology

[0002] Apple harvesting is an important agricultural process involving the collection of ripe apples from trees. Apple harvesting robots are commonly used in this process; these are automated devices designed to improve harvesting efficiency and reduce labor costs. Utilizing advanced technology and algorithms, these robots can autonomously identify, pick, and collect ripe apples in orchards. Often based on image recognition, most apple harvesting robots are equipped with cameras and sensors to identify ripe apples. These vision systems analyze the apples' color, shape, and size to determine their ripeness. They also possess robotic arms that can mimic human hand movements, easily picking the fruit without damaging the plant or the fruit.

[0003] Traditional apple-picking robots rely solely on image recognition to determine apple ripeness, which is a rather simplistic approach. This is especially problematic in harsh weather conditions such as rain or snow, which can affect the accuracy of image recognition and lead to inaccurate judgments, potentially resulting in the picking of unripe apples.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and device for determining the maturity of apples in an apple-picking robot, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for classifying and determining the ripeness of apples for apple-picking robots, comprising the following steps:

[0008] S1. Near-infrared spectroscopy is used to collect reflectance spectral parameters under different wavelengths. The reflectance spectral parameters include whiteboard reflectance spectral current intensity, dark current intensity, and apple reflectance spectral current intensity.

[0009] S2. Perform correlation analysis on the reflectance spectral parameters to generate absorbance. Analyze the absorbance to generate the apple sugar content index (PTH). The apple sugar content index (PTH) is used to reflect the sugar content of the target apple.

[0010] S3. Collect pressure data to reflect the softness and hardness of the target apple;

[0011] S4. Perform correlation analysis on the hardness data to generate the average pressure F and pressure difference S, and perform correlation analysis on the average pressure F and pressure difference S to generate the apple hardness index H. The apple hardness index H is used to reflect the hardness of the target apple.

[0012] S5. Obtain sample apples, use an apple picking robot to photograph the sample apples, and obtain reference images and actual images;

[0013] S6. Perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate Tone Maturity Dynamic Difference (TSM). The Tone Maturity Dynamic Difference (TSM) is used to reflect the degree of deviation between the actual grayscale and the reference grayscale when the apple picking robot performs image recognition on the apple.

[0014] S7. Correlation analysis was performed on the apple sugar content index PTH, apple firmness index H, and color ripeness dynamic difference TSM to generate the apple maturity assessment index PCP, which is used to reflect the ripeness of apples.

[0015] S8. Set a threshold, compare the apple maturity assessment index PCP with the threshold, and output the apple maturity level.

[0016] Furthermore, in S1, the values ​​of different wavelengths λ are 970nm, 1200nm, and 1450nm.

[0017] Furthermore, the whiteboard reflectance spectral current intensity is the current intensity used to reflect the reflectance spectrum at different wavelengths when the near-infrared spectral light source is turned on and no test object is placed in it; the dark current intensity is the current intensity used to reflect the reflectance spectrum when the near-infrared spectral light source is not turned on; and the apple reflectance spectral current intensity is the current intensity used to reflect the reflectance spectrum at different wavelengths when the near-infrared spectral light source is turned on and the target apple is measured.

[0018] The correlation analysis of the three reflectance spectral parameters is performed as follows: the dark current intensity is subtracted from the reflectance spectral current intensity of the apple, and the dark current intensity is subtracted from the reflectance spectral current intensity of the white board. Finally, the difference between the former and the latter is divided to obtain the relative reflectance. For the calculated relative reflectance, its reciprocal is calculated first, and then the logarithm of the reciprocal is taken. The absorbance is obtained through these two steps.

[0019] The obtained relative reflectance eliminates the influence of ambient light and is used to reflect the reflectance of the target apple surface at different wavelengths. The obtained absorbance reflects the absorbance of the target apple at different wavelengths. First, the absorbance corresponding to three specific wavelengths, 970nm, 1200nm, and 1450nm, is selected. A corresponding weighting coefficient is matched to the absorbance of each wavelength, with the weighting coefficient for 970nm being greater than that for 1200nm, and the weighting coefficient for 1200nm being greater than that for 1450nm. The absorbance of each wavelength is multiplied by its own weighting coefficient to obtain three product results. Then, these three product results are added together to obtain the sum, which is the apple sugar content index (PTH), reflecting the sugar content of the target apple.

[0020] Furthermore, in step S3, the pressure data reflecting the hardness of the target apple is collected as follows: a compression sensor is used to press the surface of the target apple and apply pressure. When the displacement change of the sensor reaches 1.5mm, the pressure value at this time is collected. During the collection process, eight positions are evenly selected on the surface of the target apple, and the pressure at each position is measured to obtain the pressure data of each of the eight positions.

[0021] Add up the pressure data from all eight locations to obtain the total pressure. Divide this total by 8 to calculate the average pressure, which reflects the uniformity of pressure on the target apple's surface. Calculate the difference between the pressure data at each location and the average pressure, and then square each difference. Add all the squared results to obtain the sum of squares. Divide this sum of squares by 8 to obtain the mean square value. Take the square root of the mean square value to obtain the pressure difference, which reflects the uniformity of pressure on the target apple's surface. Divide the average pressure by 1.5 to obtain a partial pressure result. Add this partial pressure result to the pressure difference to obtain the apple hardness index H, which reflects the softness or hardness of the target apple.

[0022] Furthermore, in step S5, an apple is taken as a sample apple, and a reference image is obtained by photographing the sample apple in a well-lit indoor environment. Alternatively, a sample apple can be photographed in an apple orchard to obtain an actual image.

[0023] Furthermore, for any pixel in the reference image, the same weighted average grayscale method as the reference image is used for its red, green, and blue color components. The red, green, and blue components of the pixel are assigned weights of 0.6, 0.2, and 0.2, respectively. The grayscale of the pixel is obtained by weighted calculation and recorded as the actual image grayscale, reflecting the grayscale level of the pixel in the actual image.

[0024] The total number of rows and columns of pixels in the actual image is counted. The gray values ​​of all pixels in the actual image are summed to obtain the total gray value of the actual image. The total gray value is divided by the total number of pixels to obtain the overall average gray value of the actual image, which is denoted as the actual gray value, reflecting the overall average gray value characteristics of the actual image.

[0025] Using the actual grayscale as a benchmark, the difference between the actual grayscale and the reference grayscale is first calculated. Then, the ratio of this difference to the actual grayscale is calculated to obtain the Tone Maturity Dynamic Difference (TSM), which quantitatively reflects the degree of deviation between the actual grayscale and the reference grayscale when the apple picking robot performs image recognition of apples in an orchard environment.

[0026] Furthermore, correlation analysis was conducted on the apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate the apple ripeness assessment index (PCP), based on the following formula:

[0027]

[0028] The Apple Maturity Assessment Preference (PCP) is used to reflect the degree of ripeness of apples.

[0029] Furthermore, when PCP ≥ θ, the output apple ripeness level is Level 1, at which point the target apple can be picked; when PCP < θ, the output apple ripeness level is Level 2, at which point the target apple cannot be picked.

[0030] The present invention also provides a target ripeness determination and classification system for apple picking robots, the system being used to execute the above-described target ripeness determination and classification method for apple picking robots, comprising:

[0031] The reflectance spectral parameter acquisition module is used to acquire reflectance spectral parameters at different wavelengths using near-infrared spectroscopy.

[0032] The reflectance spectral parameter analysis module is used to perform correlation analysis on reflectance spectral parameters, generate absorbance, analyze absorbance, and generate apple sugar content index (PTH).

[0033] Apple firmness data acquisition module, used to collect firmness data of target apples;

[0034] The softness and hardness data analysis module is used to perform correlation analysis on softness and hardness data, generate equal pressure F and pressure difference S, and perform correlation analysis on equal pressure F and pressure difference S to generate apple softness and hardness index H.

[0035] The environmental feature acquisition module is used to acquire sample apples, use an apple picking robot to photograph the sample apples, and acquire reference images and actual images.

[0036] The image analysis module is used to perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate hue intensity dynamic difference (TSM).

[0037] The comprehensive analysis module is used to perform correlation analysis on apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate apple maturity assessment index (PCP), which reflects the degree of apple ripeness.

[0038] The output module compares the apple maturity assessment index PCP with the threshold and outputs the apple maturity level.

[0039] The present invention also provides an apple harvesting robot target ripeness determination and classification device, the device including a storage medium and a processor, wherein the storage medium stores computer program instructions, and the computer program instructions are executed by the processor to implement the above-mentioned apple harvesting robot target ripeness determination and classification method.

[0040] This invention collects reflectance spectral parameters through near-infrared spectroscopy and performs correlation analysis to reduce the influence of environmental factors, generating an apple sugar content index to reflect the sugar content of the target apple. Simultaneously, it collects and analyzes apple firmness data to generate an apple firmness index to reflect the firmness of the target apple. Furthermore, it analyzes the deviation of the actual grayscale reference grayscale in image recognition to generate a hue ripeness dynamic difference value to reflect the deviation of the actual grayscale reference grayscale when the apple-picking robot performs image recognition on the apple. Combining the apple sugar content index, apple firmness index, and hue ripeness dynamic difference analysis, the invention finally outputs an apple ripeness assessment index to determine whether the apple is ready for harvest. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall method flow of the present invention;

[0042] Figure 2 This is a schematic diagram of the three-wavelength absorbance of the present invention;

[0043] Figure 3 This is a schematic diagram illustrating the correlation between PCP and Brix values ​​in this invention;

[0044] Figure 4 This is a schematic diagram illustrating the correlation between the Brix value and 10 times TSM in this invention;

[0045] Figure 5 This is a schematic diagram illustrating the relationship between Brix values ​​and PTH in this invention;

[0046] Figure 6 This is a schematic diagram illustrating the relationship between Brix values ​​and H in this invention;

[0047] Figure 7 This is a schematic diagram of the overall system modules of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0049] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0050] Example:

[0051] Please see Figures 1 to 6 The present invention provides a technical solution:

[0052] A method for classifying and determining the ripeness of apples for apple-picking robots, comprising the following steps:

[0053] Step 1: Near-infrared spectroscopy was used to collect reflectance spectral parameters at different wavelengths. These parameters included the reflectance spectral current intensity of the white board, the dark current intensity, and the reflectance spectral current intensity of the apple. The wavelengths λ were 970 nm, 1200 nm, and 1450 nm. At wavelengths of 970 nm, 1200 nm, and 1450 nm, the spectral characteristics interacted strongly with the sugar content, allowing for better measurement of sugar content. For apples, higher ripeness generally indicates higher sugar content.

[0054] Step 2: Perform correlation analysis on the reflectance spectral parameters to generate absorbance. Analyze the absorbance to generate the apple sugar content index (PTH). The apple sugar content index (PTH) is used to reflect the sugar content of the target apple.

[0055] Whiteboard Reflectance Spectral Current Intensity I white(λ) is used to reflect the reflected spectral current intensity at different wavelengths λ when the near-infrared spectral light source is activated and no test object is placed in the sample; dark current intensity I dark Used to reflect the intensity of the reflected spectral current when the near-infrared spectral light source is not activated; Apple reflected spectral current intensity I nir (λ) is used to reflect the intensity of the reflected spectral current at different wavelengths λ when measuring the target apple with the near-infrared spectral light source activated; correlation analysis is performed on the reflected spectral parameters to generate the relative reflectance R. nir (λ), based on the following formula:

[0056]

[0057] Relative reflectivity R nir (λ) is used to reflect the surface reflectance of apples at different wavelengths λ after eliminating the influence of ambient light;

[0058] For relative reflectivity R nir (λ) Perform correlation analysis to generate absorbance A λ The formula used is:

[0059]

[0060] Absorbance A λ This is used to reflect the absorbance of a target apple at different wavelengths λ. There is a direct linear relationship between absorbance and sugar content. When measuring sugar content using near-infrared spectroscopy, the greater the absorbance of the apple, the higher its sugar concentration.

[0061] Absorbance A λ Correlation analysis was performed to generate the apple sugar content index (PTH), based on the following formula:

[0062] PTH=α1*A 970 +α2*A 1200 +α3*A 1450

[0063] The apple sugar content index (PTH) reflects the sugar content of a target apple. α1 is the absorbance weighting coefficient for the target apple at 970 nm, α2 is at 1200 nm, and α3 is at 1450 nm, with α1 > α2 > α3. The absorbance weighting coefficients for these three wavelengths (970 nm, 1200 nm, and 1450 nm) are typically determined based on their respective absorption characteristics for sugars, signal intensity, and interference with other components. 970 nm is a wavelength where sugars (such as glucose and fructose) have strong absorption and is generally considered important in sugar content measurements. Because this wavelength is closely related to the characteristic absorption frequency of sugar, it is given a high weight. The 1200nm wavelength is related to the absorption of various components, including sugars, fats and water. Although its absorption characteristics for sugars are not as obvious as those for 970nm, it still provides useful information and is usually given a medium weight in the model. The 1450nm wavelength is mainly related to the absorption characteristics of water. Although it has less direct absorption of sugars, it is given a certain weight to correct for the influence of water on sugar content measurement. Table 1 shows the statistics of absorbance at each wavelength for 20 groups of unripe and 20 groups of ripe apples. The summary table of absorbance at each wavelength is generated after the data is compiled.

[0064] Table 1 Summary of Absorbance at Each Wavelength

[0065]

[0066]

[0067] like Figure 2As shown, for both ripe and unripe apples, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm for ripe apples shows an overall increasing trend across all three wavelengths. Calculations using the above formula indicate that the PTH of ripe apples is generally higher than that of unripe apples, validating the rationality of using PTH to quantify sugar content changes. For example, setting α1 = 6.75, α2 = 5.25, and α3 = 3.00, for a specific unripe apple, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm were measured to be 0.3, 0.2, and 0.1, respectively. After processing the apple... The formula used to calculate the PTH (Premium Sugar Content Index) is 3.4 (rounded to one decimal place). For a ripe apple, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm was measured to be 1.2, 1.0, and 0.8, respectively, resulting in a final calculation of 15.8. For apples with even higher ripeness, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm was measured to be 1.8, 1.6, and 1.4, respectively, resulting in a final calculation of 24.8. As the degree of ripeness increases, the PTH calculation results generally show a positive correlation.

[0068] Step 3: Collect pressure data to reflect the hardness and softness of the target apple;

[0069] Step 4: Perform correlation analysis on the hardness data to generate the average pressure F and pressure difference S, and perform correlation analysis on the average pressure F and pressure difference S to generate the apple hardness index H. The apple hardness index H is used to reflect the hardness of the target apple.

[0070] In step S3, the pressure data reflecting the hardness of the target apple is collected as follows: a compression sensor is pressed against the surface of the target apple, and pressure is applied. When the displacement change is 1.5 mm, the pressure is collected. During pressure collection, eight locations on the surface of the target apple are measured evenly to obtain the pressure F. p Let p be the index of the target apple's surface location, and p∈[1,8], for pressure F p Correlation analysis was performed to generate the equal pressure F, based on the following formula:

[0071]

[0072] The uniform pressure F reflects the uniform pressure value on the surface of the target apple. Correlation analysis is performed on the uniform pressure F to generate the pressure difference S, based on the following formula:

[0073]

[0074] The pressure difference S is used to reflect the uniformity of pressure on the surface of the target apple. Even if the average pressure is the same, the more uneven the pressure distribution, i.e., the larger S is, the more unstable the apple texture is and the harder the overall hardness characteristics are.

[0075] A correlation analysis was performed on the uniform pressure F and the pressure difference S to generate the apple firmness index H. The formula used was as follows:

[0076]

[0077] Among them, the apple hardness index H is used to reflect the hardness of the target apple, and the average pressure F is the average pressure at 8 locations on the apple surface, reflecting the average pressure on the apple surface. From a physical point of view, the harder the apple, the greater the pressure required to press it to the same displacement. Therefore, the size of F is directly related to the hardness of the apple. The larger the F, the greater the force required to press the apple to the target displacement, that is, the harder the apple is, which in turn affects the size of H.

[0078] A correlation analysis was performed on the uniform pressure F and the pressure difference S to generate the apple firmness index H. The formula used was as follows:

[0079]

[0080] The apple firmness index H reflects the firmness of the target apple. The higher the apple firmness index H, the firmer the apple. Apples that are more mature are less firm. Actual measurements show that indenting the apple surface by 1.5mm typically requires a force of 5N-15N. For example, for an unripe apple, we take the highest value F=15 and S=0. Using the formula above, the apple firmness index H is 10.00. For a ripe apple, we take the minimum value F=5 and S=0, rounded to two decimal places, resulting in 3.33.

[0081] Step 5: Obtain sample apples. Use the apple picking robot to photograph the sample apples and obtain reference images and actual images. In step S5, take an apple as a sample apple, photograph the sample apple in well-lit indoor conditions and obtain a reference image, and photograph the sample apple in the orchard and obtain an actual image.

[0082] Step 6: Perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate Tone Maturity Dynamic Difference (TSM). The Tone Maturity Dynamic Difference (TSM) is used to reflect the degree of deviation between the actual grayscale and the reference grayscale when the apple picking robot performs image recognition on apples.

[0083] The red, green, and blue components of the pixel in the i-th row and j-th column of the reference image are represented by R. c (i,j), G c(i,j), B c (i,j) indicates that correlation analysis is performed on the sample to generate a reference image grayscale value I. c (i,j), based on the following formula:

[0084] I c (i,j)=0.299*R c (i,j)+0.587*G c (i,j)+0.114*B c (i,j)

[0085] Reference image grayscale I c (i,j) is used to reflect the gray level of the pixel in the i-th row and j-th column of the reference image;

[0086] For the reference image grayscale I c Perform correlation analysis on (i,j) to generate reference grayscale I. c The formula used is:

[0087]

[0088] Among them, reference grayscale I c Used to reflect the average gray level of the reference image, M is the number of rows of pixels in the captured image, and N is the number of columns of pixels in the captured image;

[0089] In a real image, the red, green, and blue components of the pixel in the i-th row and j-th column are represented by R. s (i,j), G s (i,j), B s (i,j) represents the actual image grayscale value I generated by performing correlation analysis. s (i,j), based on the following formula:

[0090] I s (i,j)=0.6*R s (i,j)+0.2*G s (i,j)+0.2*B s (i,j)

[0091] Actual image grayscale I s (i,j) is used to reflect the gray level of the pixel in the i-th row and j-th column of the actual image;

[0092] For actual image grayscale I s Perform correlation analysis on (i,j) to generate the actual grayscale value I. c The formula used is:

[0093]

[0094] Among them, actual gray level Is Used to reflect the average gray level of the actual image;

[0095] Changes in apple ripeness directly lead to changes in skin pigmentation. In the unripe stage, the skin is dark green or light green, with strong light absorption and weak light reflection, resulting in darker pixels in the image. In the ripe stage, the skin exhibits red, yellow, and orange color characteristics, with a significant decrease in light absorption and a strong reflective ability. When light shines on the skin of a ripe apple, more light is reflected into the camera lens, resulting in brighter pixels in the image. Therefore, the calculated average grayscale gradually increases with ripeness.

[0096] For actual grayscale I s and reference grayscale I c Correlation analysis was performed to generate the Tone Maturity Dynamics (TSM), based on the following formula:

[0097]

[0098] The Tone Maturity Dynamic Difference (TSM) reflects the actual grayscale state of an apple. It is the relative deviation from the target harvest maturity standard grayscale. When TSM < 0, it indicates that the actual apple's grayscale is lower than the target maturity standard, reflecting that the apple is unripe. When TSM is close to 0, it indicates that the actual apple's grayscale is close to the target maturity standard, reflecting that the apple has reached or is close to the ideal harvest maturity. When TSM > 0, it indicates that the actual apple's grayscale is higher than the standard of a ripe apple in the reference image. For example, for an unripe apple with a greenish surface, its grayscale is 80, the reference grayscale is 150, and TSM < 0. For a ripe apple with a reddish surface, its grayscale is 170, the reference grayscale is 150, and TSM = 0.12 (rounded to two decimal places).

[0099] Step 7: Perform correlation analysis on apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate apple maturity assessment index (PCP). The apple maturity assessment index (PCP) is used to reflect the ripeness of apples.

[0100] Correlation analysis was performed on the apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate the apple ripeness assessment index (PCP). The formula used is as follows:

[0101]

[0102] The Apple Maturity Assessment Index (PCP) reflects the ripeness of apples. It is a quantitative indicator that integrates sugar content, firmness, and the deviation from the actual grayscale reference grayscale value recognized by robots. A higher PCP indicates a higher degree of ripeness in the target apple, avoiding reliance on a single dimension. TSM and PTH increase with apple ripeness, while H decreases with ripeness. However, the inverse correlation between H and ripeness does not affect the increasing PCP with ripeness. The formula can be broken down as follows:

[0103]

[0104] H is a positive contributor, while H is a negative contributor. Their trends with maturity are completely opposite. Numerical verification shows that the increase in the positive contributor is much greater than the decrease in the negative contributor, ultimately driving an overall increase in PCP. For apples with low maturity, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm are 0.3, 0.2, and 0.1, respectively. The calculated result using the formula for generating the apple sugar content index PTH is 3.4, and H takes its maximum value of 10.00. For apples at the most common ripeness level, ready for picking, the absorbance at wavelengths of 970nm, 1200nm, and 1450nm was measured to be 1.2, 1.0, and 0.8, respectively. The final calculated result was 15.8, and H was taken as the minimum value of 3.33. The decrease in H was 6.60. Therefore, H decreased with maturity, but the decrease was less than the increase in the positive contribution item. The function of the 10 item is to scale the values, which preserves the relative relationship of the original data and enhances the intuitiveness and distinguishability of the indicator.

[0105] Step 8: Set a threshold, compare the apple maturity assessment index PCP with the threshold, and output the apple maturity level.

[0106] Set a threshold θ, where θ>0. Adjust the threshold according to the actual situation based on the different types of apples. For example, if the threshold θ is set to 0.5, when PCP≥θ, output the apple ripeness level as Level 1, indicating that the target apple can be picked; when PCP<θ, output the apple ripeness level as Level 2, indicating that the apple is not ripe.

[0107] The core physiological change in apple ripening is the conversion of starch into sugar, which leads to an increase in soluble solids content. Brix value is significantly positively correlated with maturity. This association has become a common understanding in the fields of food science, agricultural planting, and postharvest processing of fruits and vegetables. Table 1 shows that the known Brix value is selected as the reference standard for actual maturity, i.e., the degree of maturity. Under the premise of setting α1=10, α2=6, α3=4, 40 sets of PTH, H, TSM data were collected. PCP was calculated based on the above formula and compared with θ to output the result of whether it is ready for picking.

[0108] Table 1. Verification Table for Apple Maturity Assessment Index

[0109]

[0110]

[0111] like Figures 3-6 As shown, Figure 3 Both curves show an upward trend, indicating that PCP increases with apple ripeness, thus verifying the effectiveness of PCP as a measure of apple ripeness. Figure 4 In the image, the right vertical axis represents TSM magnified 10 times. TSM reflects the RGB grayscale of an apple, and negative values ​​clearly indicate an immature stage. In samples 38, 39, and 40, the fruit is overripe, with the apple color gradually darkening and turning brown, consistent with the pattern of increasing Brix value and decreasing TSM. Figure 3 The fact that the values ​​of samples 38, 39, and 40 no longer gradually increase matches the pattern suggests that TSM can reflect quality to some extent. Figure 5 and Figure 6 The data clearly shows that as the degree of ripeness increases, PTH (particulate matter content) increases due to increased sugar content, while PTH decreases due to decreased hardness.

[0112] Reference Figure 7 The present invention also provides a target maturity determination and classification system for apple picking robots, the system being used to execute the above-described target maturity determination and classification method for apple picking robots, comprising:

[0113] The reflectance spectral parameter acquisition module is used to acquire reflectance spectral parameters at different wavelengths using near-infrared spectroscopy.

[0114] The reflectance spectral parameter analysis module is used to perform correlation analysis on reflectance spectral parameters, generate absorbance, analyze absorbance, and generate apple sugar content index (PTH).

[0115] Apple firmness data acquisition module, used to collect firmness data of target apples;

[0116] The softness and hardness data analysis module is used to perform correlation analysis on softness and hardness data, generate equal pressure F and pressure difference S, and perform correlation analysis on equal pressure F and pressure difference S to generate apple softness and hardness index H.

[0117] The environmental feature acquisition module is used to acquire sample apples, use an apple picking robot to photograph the sample apples, and acquire reference images and actual images.

[0118] The image analysis module is used to perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate hue intensity dynamic difference (TSM).

[0119] The comprehensive analysis module is used to perform correlation analysis on apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate apple maturity assessment index (PCP), which reflects the degree of apple ripeness.

[0120] The output module compares the apple maturity assessment index PCP with the threshold and outputs the apple maturity level.

[0121] An apple harvesting robot target ripeness determination and classification device is provided. The device includes a storage medium and a processor. The storage medium stores computer program instructions. When the computer program instructions are executed by the processor, they implement the apple harvesting robot target ripeness determination and classification method according to any one of claims 1-8.

[0122] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0123] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0124] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0125] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for classifying and determining the ripeness of apples for a harvesting robot, characterized in that, The specific steps include: S1. Near-infrared spectroscopy is used to collect reflectance spectral parameters under different wavelengths. The reflectance spectral parameters include whiteboard reflectance spectral current intensity, dark current intensity, and apple reflectance spectral current intensity. S2. Perform correlation analysis on the reflectance spectral parameters to generate absorbance. Analyze the absorbance to generate the apple sugar content index (PTH). The apple sugar content index (PTH) is used to reflect the sugar content of the target apple. S3. Collect pressure data to reflect the softness and hardness of the target apple; S4. Perform correlation analysis on the hardness data to generate the average pressure F and pressure difference S, and perform correlation analysis on the average pressure F and pressure difference S to generate the apple hardness index H. The apple hardness index H is used to reflect the hardness of the target apple. S5. Obtain sample apples, use an apple picking robot to photograph the sample apples, and obtain reference images and actual images; S6. Perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate Tone Maturity Dynamic Difference (TSM). The Tone Maturity Dynamic Difference (TSM) is used to reflect the degree of deviation between the actual grayscale and the reference grayscale when the apple picking robot performs image recognition on the apple. S7. Correlation analysis was performed on the apple sugar content index PTH, apple firmness index H, and color ripeness dynamic difference TSM to generate the apple maturity assessment index PCP, which is used to reflect the ripeness of apples. S8. Set a threshold, compare the apple maturity assessment index PCP with the threshold, and output the apple maturity level.

2. The method for determining and classifying the ripeness of apples harvesting robots according to claim 1, characterized in that: In S1, the wavelength λ takes the values ​​of 970nm, 1200nm, and 1450nm.

3. The method for determining and classifying the ripeness of apples harvesting robots according to claim 2, characterized in that: The whiteboard reflectance spectral current intensity is used to reflect the reflectance spectrum at different wavelengths when the near-infrared spectral light source is turned on and no test object is placed in it. The dark current intensity is used to reflect the reflectance spectrum when the near-infrared spectral light source is not turned on. The apple reflectance spectral current intensity is used to reflect the reflectance spectrum at different wavelengths when the near-infrared spectral light source is turned on and the target apple is measured. The correlation analysis of the three reflectance spectral parameters is performed as follows: the dark current intensity is subtracted from the reflectance spectral current intensity of the apple, and the dark current intensity is subtracted from the reflectance spectral current intensity of the white board. Finally, the difference between the former and the latter is divided to obtain the relative reflectance. For the calculated relative reflectance, its reciprocal is calculated first, and then the logarithm of the reciprocal is taken. The absorbance is obtained through these two steps. The obtained relative reflectance eliminates the influence of ambient light and is used to reflect the reflectance of the target apple surface at different wavelengths. The obtained absorbance reflects the absorbance of the target apple at different wavelengths. First, the absorbance corresponding to three specific wavelengths, 970nm, 1200nm, and 1450nm, is selected. A corresponding weighting coefficient is matched to the absorbance of each wavelength, with the weighting coefficient for 970nm being greater than that for 1200nm, and the weighting coefficient for 1200nm being greater than that for 1450nm. The absorbance of each wavelength is multiplied by its own weighting coefficient to obtain three product results. Then, these three product results are added together to obtain the sum, which is the apple sugar content index (PTH), reflecting the sugar content of the target apple.

4. The method for determining and classifying the ripeness of apples harvesting robots according to claim 1, characterized in that: In step S3, the pressure data reflecting the hardness of the target apple is collected as follows: a compression sensor is used to press the surface of the target apple and apply pressure. When the displacement of the sensor reaches 1.5mm, the pressure value at this time is collected. During the collection process, eight positions are evenly selected on the surface of the target apple, and the pressure at each position is measured to obtain the pressure data of each of the eight positions. Add up the pressure data from all 8 locations to get the total pressure. Divide the total by 8 to calculate the average pressure, which reflects the uniformity of the pressure on the surface of the target apple. Calculate the difference between the pressure data at each location and the average pressure, and then square each difference. Add up all the squared results to get the sum of squares. Divide the sum of squares by 8 to get the mean square value. The square root of the mean square value is taken to obtain the pressure difference, which reflects the uniformity of pressure at various locations on the surface of the target apple. The average pressure is divided by 1.5 to obtain a partial pressure result. This partial pressure result is added to the pressure difference to obtain the apple firmness index H, which reflects the firmness of the target apple.

5. The method for determining and classifying the ripeness of apples harvesting robots according to claim 1, characterized in that: In step S5, an apple is taken as a sample apple, and a reference image is obtained by photographing the sample apple in a well-lit indoor environment. The sample apple is then photographed in an apple orchard to obtain an actual image.

6. The method for determining and classifying the ripeness of apples harvesting robots according to claim 5, characterized in that: For any pixel in the reference image, the same weighted average grayscale method as the reference image is used for its red, green and blue color components. The red, green and blue components of the pixel are assigned weights of 0.6, 0.2 and 0.2 respectively. The grayscale of the pixel is obtained by weighted calculation and recorded as the grayscale of the actual image, which reflects the grayscale level of the pixel in the actual image. The total number of rows and columns of pixels in the actual image is counted. The gray values ​​of all pixels in the actual image are summed to obtain the total gray value of the actual image. The total gray value is divided by the total number of pixels to obtain the overall average gray value of the actual image, which is denoted as the actual gray value, reflecting the overall average gray value characteristics of the actual image. Using the actual grayscale as a benchmark, the difference between the actual grayscale and the reference grayscale is first calculated. Then, the ratio of this difference to the actual grayscale is calculated to obtain the Tone Maturity Dynamic Difference (TSM), which quantitatively reflects the degree of deviation between the actual grayscale and the reference grayscale when the apple picking robot performs image recognition of apples in an orchard environment.

7. The method for determining and classifying the ripeness of apples harvesting robots according to claim 1, characterized in that: Correlation analysis was performed on the apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate the apple ripeness assessment index (PCP). The formula used is as follows: The Apple Maturity Assessment Preference (PCP) is used to reflect the degree of ripeness of apples.

8. The method for determining and classifying the ripeness of apples harvesting robots according to claim 1, characterized in that: When PCP ≥ θ, the apple's maturity level is output as Level 1, and the target apple can be picked. When PCP < θ, the apple's maturity level is output as Level 2, and the target apple cannot be picked.

9. A target ripeness determination and classification system for an apple picking robot, characterized in that: The system is used to execute the apple harvesting robot target ripeness determination and classification method according to any one of claims 1-8, including: The reflectance spectral parameter acquisition module is used to acquire reflectance spectral parameters at different wavelengths using near-infrared spectroscopy. The reflectance spectral parameter analysis module is used to perform correlation analysis on reflectance spectral parameters, generate absorbance, analyze absorbance, and generate apple sugar content index (PTH). Apple firmness data acquisition module, used to collect firmness data of target apples; The softness and hardness data analysis module is used to perform correlation analysis on softness and hardness data, generate equal pressure F and pressure difference S, and perform correlation analysis on equal pressure F and pressure difference S to generate apple softness and hardness index H. The environmental feature acquisition module is used to acquire sample apples, use an apple picking robot to photograph the sample apples, and acquire reference images and actual images. The image analysis module is used to perform correlation analysis on the reference image to generate reference grayscale, perform correlation analysis on the actual image to generate actual grayscale, and perform correlation analysis on the reference grayscale and actual grayscale to generate hue intensity dynamic difference (TSM). The comprehensive analysis module is used to perform correlation analysis on apple sugar content index (PTH), apple firmness index (H), and color ripeness dynamic difference (TSM) to generate apple maturity assessment index (PCP), which reflects the degree of apple ripeness. The output module compares the apple maturity assessment index (PCP) with a threshold and outputs the apple's maturity level.

10. A target ripeness determination and classification device for an apple picking robot, characterized in that: The device includes a storage medium and a processor. The storage medium stores computer program instructions, which, when executed by the processor, implement the target maturity determination and classification method for apple picking robots according to any one of claims 1-8.

Images