Image vision-based litchi recognition and yield estimation method, device and medium

Abstract

The present application relates to a kind of litchi identification and yield estimation method, equipment and medium based on image vision, method includes obtaining orchard litchi fruit tree contour, litchi fruit etc. Image data as training data, pre-processing is carried out;Litchi fruit identification is carried out to the Haar-like feature in matrix feature after pre-processing training data;The data of the image after identification are input into litchi volume prediction model, are deduced using multilayer perceptron, obtain connection weight;Using the litchi volume prediction model after training to carry out volume prediction to the data to be predicted;The yield of orchard litchi is obtained by adding the predicted volume.This application obtains orchard litchi fruit tree contour, litchi fruit etc. Image data as prediction input value, uses AdaBoost algorithm to matrix feature as weak classifier, constructs strong classifier, obtains accurate prediction volume of litchi using error back propagation algorithm, lays foundation for the relationship between weather, wind, fertilization and yield.

Description

Technical Field

[0001] This invention belongs to the field of crop image recognition and yield estimation technology, specifically relating to a method, device and medium for litchi recognition and yield estimation based on image vision. Background Technology

[0002] Lychee is a tropical and subtropical fruit, beloved for its unique flavor and nutritional value. However, its production and management require significant human and resource investment, particularly in lychee identification and yield estimation. Traditional methods often rely on manual observation and recording, which is not only time-consuming and labor-intensive but also limited in accuracy by human experience and subjective judgment. Therefore, there is an urgent need for an efficient and accurate method for lychee identification and yield estimation to improve production efficiency and management levels.

[0003] Existing litchi identification methods primarily focus on target detection of the litchi fruit, mainly utilizing image data of litchi trees for basic image recognition to support biomimetic litchi harvesting. However, these methods are not deep or comprehensive enough in terms of litchi yield estimation and prediction, failing to provide strong evidence for litchi harvesting. How to comprehensively analyze litchi image data to achieve accurate litchi fruit detection and yield estimation, and establish a complete litchi identification and yield estimation method and system, is a pressing problem that needs to be solved in the current digital age for precision harvesting in smart agriculture. Summary of the Invention

[0004] The purpose of this invention is to provide a method, device, and medium for litchi identification and yield estimation based on image vision. By combining computer vision, machine learning, and image processing technologies, it aims to solve the problem of comprehensively analyzing litchi image data to achieve accurate detection and yield estimation of litchi fruits, thereby significantly improving the efficiency and accuracy of litchi planting and management.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A method for litchi identification and yield estimation based on image vision includes the following steps:

[0007] A. Obtain the outline of the lychee trees in the orchard, the image data of the lychee fruits, and the volume data of the lychees as training data;

[0008] B. Perform corresponding preprocessing on the training data according to the data type;

[0009] C. Using the preprocessed litchi image training data and prediction data, litchi fruit identification is performed based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to use the matrix features as a weak classifier to construct a strong classifier in order to obtain the litchi identification region.

[0010] D. Input the data of the identified image into the litchi volume prediction model, and use a multilayer perceptron to derive and determine the connection weights on the unit path of the litchi volume prediction model.

[0011] E. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and sum the predicted volumes to obtain the litchi yield of the orchard.

[0012] Further, step B specifically involves: sequentially adjusting the brightness, performing median filtering, and adjusting the contrast of the captured lychee tree image to obtain the lychee tree outline image.

[0013] Further, in step C, lychee fruit recognition is performed based on the Haar-Like features in the matrix features. Specifically, a search window is set up, and the feature value is equal to the sum of the pixel values ​​in the black area of ​​the matrix minus the sum of the pixel values ​​in the white area. Here, the matrix area is a detector with an m×m pixel resolution. The black and white areas are divided based on four features: edge features, linear features, center features, and diagonal features, respectively, for lychee fruit recognition.

[0014] Furthermore, step C also includes: based on the litchi identification region, using sequence values ​​to label litchi fruits in different regions of the image to obtain a label matrix, and converting the RGB image into an HSV image, and using Gaussian filtering + Canny high-threshold edge detection to remove small isolated regions.

[0015] Further, step D includes the following steps:

[0016] D1. Assuming that lychees are ideally symmetrical and their images are elliptical, find the longest side of the two-dimensional image of a lychee based on the pixels. Using the longest side as the X-axis and the center of the X-axis as the Y-axis, rely on formula a... 2 =b 2 +c 2 and Substitute the values ​​into the equations to find the values ​​of a, b, and c respectively.

[0017] D2. For identifying the lychee region, perform horizontal and vertical pixel overlay. Based on the pixel accumulation in the horizontal and vertical directions, obtain the number of minimum values ​​d. This value can determine the number of lychees in the image.

[0018] D3. Obtain the values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above lychee image as training input values, and use the volume of the lychee as the training output value. Input data is input from the input layer, passes through the intermediate layer, and is finally output from the output layer. By comparing the actual output and the expected output, the error signal is obtained. The error signal is propagated from the output layer forward layer by layer to obtain the error signal of each layer. Then, the connection weights of each layer are adjusted to reduce the error.

[0019] Furthermore, in step D3, the weights are adjusted using gradient descent to continuously adjust the weights to minimize the error, thereby learning the optimal connection weights. Specifically:

[0020] Assuming the litchi image is an ellipsoid, and obtaining the corresponding parameters a, b, c, d, these four parameters are expressed as x1, x2, x3, x4, respectively. y is the output, and the derivation is performed using a multilayer perceptron. w 1ij w represents the connection weights between the input layer and the intermediate layer. 2j1 This represents the connection weight between the intermediate visible layer and the output layer, where i represents the input unit and j represents the intermediate layer unit;

[0021] D31. Adjust the connection weights between the intermediate layer and the output layer, and adjust the error function E relative to the connection weights w. 2ij Differentiate and expand to find the derivative of the composite function;

[0022]

[0023] set up Find the error function E. The derivative;

[0024]

[0025] Let g = -x, h = e g If p = 1 + h, then f(u) = p -1 According to the derivation, we can obtain:

[0026]

[0027] After simplification, we get:

[0028]

[0029] And because For w 2j1 The result of the derivative depends only on the value of the intermediate layer; therefore, let z j This represents the value of the intermediate layer. In summary, the connection weight adjustment values ​​are as follows:

[0030]

[0031]

[0032] D32. Adjust the weights of the connection layers between the input layer and the intermediate layers; the adjustment value of the connection weights between the input layer and the intermediate layers is determined based on the error function of the output layer, and the derivative formula is shown below:

[0033]

[0034] Similar to the calculation method for the weight adjustment values ​​between the intermediate and output layers, the output y is also calculated by expanding the formula and then taking the derivative of the composite function. However, here it represents the activation values ​​between the intermediate and output layer units. Connection weights between the output layer and the intermediate layer Differentiate, u 21 The value z of the intermediate layer j and connection weight w 2j1 Calculations show that for z j Differentiation yields the following equation:

[0035]

[0036] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z of the intermediate layer j Taking the derivative, the result is only related to the connection weight w. 2j1 Related;

[0037]

[0038] Below is the value z of the intermediate layer. j For connection weights Differentiate;

[0039]

[0040] Similar to y above, z j It is also the sigmoid function, with respect to z. j Taking the derivative, we get the following equation:

[0041]

[0042] Below is the activation function u between the input layer and the intermediate layer units. 1j The connection weight w between the intermediate layer and the output layer 1ij The derivative is obtained by taking the derivative and the result is only related to the input value x. i Relevant, namely:

[0043]

[0044] In summary, the derivatives of the constituent units and connection weights have been obtained.

[0045]

[0046] The connection weight w between the input layer and the intermediate layer 1ij Differentiate;

[0047]

[0048] D33. The connection weight adjustment values ​​between the intermediate layer and the output layer are shown below:

[0049]

[0050] The adjusted connection weights between the input layer and the intermediate layer are shown below:

[0051]

[0052] Further, in step E, the trained lychee volume prediction model is used to predict the volume of the data to be predicted. Specifically, the correlation between the predicted lychee volume and the actual volume is tested, and the prediction result is output if the correlation test is reasonable. That is, it is determined whether the predicted value is greater than or less than a reasonable threshold. If so, the prediction output is determined to be unreasonable. If not, the prediction output is determined to be reasonable and the volume is output.

[0053] Further, step E involves summing the predicted volumes to obtain the orchard's lychee yield, specifically including the following steps:

[0054] E1. If four cameras are used to capture images of lychee trees, and then the output volumes are summed, the estimated yield of the lychee trees can be calculated as follows:

[0055]

[0056] Where y is the volume of the r-th lychee fruit in the image taken from the j-th camera position on the i-th tree, and ρ represents the density of the lychee fruit;

[0057] E2. Use drones to capture aerial images of the lychee orchard, and obtain the pixel size L1,…,L of the aerial images of the lychee trees at a certain height h1. m Aerial photography was used to capture the entire lychee orchard (h2), and the pixel size of the aerial images was recorded as K1,…,K. m If the total pixel area is U, then the estimated lychee yield of the entire orchard is:

[0058]

[0059] A computer device, comprising:

[0060] Memory, which stores computer programs;

[0061] The processor, when executing the computer program, implements the image vision-based litchi recognition and yield estimation method.

[0062] The signal receiver connects to the camera unit and drone hangar via internal circuitry.

[0063] A computer storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the aforementioned image vision-based lychee recognition and yield estimation method.

[0064] Compared with the prior art, the beneficial effects of the present invention are:

[0065] This invention uses image data such as the outline of litchi trees and litchi fruits in orchards as prediction input values. It employs the AdaBoost algorithm to use matrix features as a weak classifier to construct a strong classifier, and uses the backpropagation algorithm to obtain accurate predicted litchi volume. This lays the foundation for studying the relationship between weather, wind, fertilization, and yield, and provides a reference for smart agriculture. By combining computer vision, machine learning, and image processing technologies, it significantly improves the efficiency and accuracy of litchi planting management. Attached Figure Description

[0066] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 The flowchart of the litchi recognition and yield estimation method based on image vision in this invention;

[0068] Figure 2 Schematic diagram of the backpropagation algorithm;

[0069] Figure 3 An internal structural diagram of a computer device;

[0070] Figure 4 Schematic diagram of litchi fruit identification based on Haar-Like features in matrix features;

[0071] Figure 5 (s,t) conditions corresponding to different rectangular features;

[0072] Figure 6a Based on the litchi identification region, the litchi fruits in different regions of the image are labeled with sequential numerical values ​​to obtain a label matrix;

[0073] Figure 6b Depend on Figure 6a A schematic diagram of coordinate system marking;

[0074] Figure 6c Mathematical idealization model. Detailed Implementation

[0075] The present invention will be further described below with reference to embodiments:

[0076] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0077] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0078] This invention provides a method for litchi identification and yield estimation based on image vision, comprising the following steps:

[0079] 1. Obtain image data of the outline of the lychee trees in the orchard, the lychee fruits, and the volume data of the lychees as training data.

[0080] 2. Perform corresponding preprocessing on the training data according to the data type.

[0081] 3. Using the preprocessed litchi image training and prediction data, litchi fruit identification is performed based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to construct a strong classifier by using the matrix features as a weak classifier.

[0082] Specifically, the process of identifying litchi fruits using Haar-Like features includes the following steps: setting up a search window, where the feature value is equal to the sum of pixel values ​​in the black area of ​​the matrix minus the sum of pixel values ​​in the white area; and setting up an m×m pixel resolution detector for the matrix area, and dividing the black and white areas based on Haar-Like features: edge features, linear features, center features, and diagonal features for litchi fruit identification.

[0083] It also includes: based on the litchi identification region, using sequence values ​​to label litchi fruits in different regions of the image to obtain a label matrix, and converting RGB images into HSV images, using Gaussian filtering + Canny high-threshold edge detection to remove small isolated regions.

[0084] 4. Input the identified image data into the litchi volume prediction model, and use a multilayer perceptron to derive and determine the connection weights on the unit paths of the litchi volume prediction model, including the following steps:

[0085] 41. Assuming that lychees are ideally symmetrical and their images are elliptical, find the longest side of the 2D image of the lychee based on the pixels. Using the longest side as the X-axis and the center of the X-axis as the Y-axis, apply formula a... 2 =b 2 +c 2 and Substitute the values ​​into the equations to find the values ​​of a, b, and c respectively.

[0086] 42. For identifying the lychee region, perform horizontal and vertical pixel overlay, and calculate the number of minimum values ​​d based on the pixel accumulation in the horizontal and vertical directions. This value can be used to determine the number of lychees in the image.

[0087] 43. Backpropagation Algorithm: The values ​​of pixels a, b, and c corresponding to the above lychee image and the number of minimum values ​​d are obtained as training input values, and the volume of the lychee is used as the training output value. The input data is input from the input layer, passes through the intermediate layer, and is finally output from the output layer. The error signal is obtained by comparing the actual output and the expected output. The error signal is propagated forward from the output layer to obtain the error signal of each layer. The error is reduced by adjusting the connection weights of each layer.

[0088] The weights are adjusted using gradient descent, continuously adjusting the weights to minimize the error, thereby learning the optimal connection weights. Specifically:

[0089] The litchi image is idealized as an ellipsoid to obtain the corresponding parameters a, b, c, d. These four parameters are then expressed as x1, x2, x3, x4, with y as the output. A multilayer perceptron is used for derivation, and w... 1ij w represents the connection weights between the input layer and the intermediate layer. 2j1 This represents the connection weights between the intermediate visible layer and the output layer. 'i' represents the input unit, and 'j' represents the intermediate layer unit.

[0090] First, adjust the connection weights between the intermediate and output layers. Adjust the error function E relative to the connection weights w. 2ij Expand the derivative into a composite function and then find its derivative.

[0091]

[0092] set up Find the error function E. The derivative of .

[0093]

[0094] Let g = -x, h = e g If p = 1 + h, then f(u) = p -1 According to the derivation, we can obtain:

[0095]

[0096] After simplification, we get:

[0097]

[0098] And because For w 2j1 The result of the derivative depends only on the value of the intermediate layer; therefore, let z j This represents the value of the intermediate layer. In summary, the connection weight adjustment values ​​are as follows:

[0099]

[0100]

[0101] Next, the weights of the connection layers between the input layer and the intermediate layers are adjusted. The adjustment values ​​of the connection weights between the input layer and the intermediate layers are determined based on the error function of the output layer, and the derivative formula is shown below.

[0102]

[0103] Similar to the calculation method for the weight adjustment values ​​between the intermediate and output layers, the output y is also calculated by expanding the formula and then taking the derivative of the composite function. The difference is that here it represents the activation values ​​between the intermediate and output layer units. Connection weights between the output layer and the intermediate layer Differentiate. u 21 The value z of the intermediate layer j and connection weight w 2j1 Calculated. For z j Differentiation yields the following equation:

[0104]

[0105] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z of the intermediate layer j Taking the derivative, the result is only related to the connection weight w. 2j1 Related.

[0106]

[0107] Below is the value z of the intermediate layer. j For connection weights Find the derivative.

[0108]

[0109] Similar to y above, z j It is also the sigmoid function, with respect to z. j Taking the derivative, we get the following equation:

[0110]

[0111] Below is the activation function u between the input layer and the intermediate layer units. 1j The connection weight w between the intermediate layer and the output layer 1ij The derivative is obtained by taking the derivative and the result is only related to the input value x. i Relevant, namely:

[0112]

[0113] In summary, the derivatives of the constituent units and connection weights have been obtained.

[0114]

[0115] The connection weight w between the input layer and the intermediate layer 1ij Find the derivative.

[0116]

[0117] Finally, the adjusted connection weights between the intermediate layer and the output layer are shown below:

[0118]

[0119] The adjusted connection weights between the input layer and the intermediate layer are shown below:

[0120]

[0121] 5. Use the trained litchi volume prediction model to predict the volume of the data to be predicted; sum the predicted volumes to obtain the litchi yield of the orchard.

[0122] The trained litchi volume prediction model is used to predict the volume of the data to be predicted. Specifically, the predicted litchi volume is compared with the actual volume, and the correlation test is performed to output a reasonable prediction result, including:

[0123] Determine whether the predicted value is greater than or less than a reasonable threshold;

[0124] If so, the predicted output is deemed unreasonable;

[0125] If not, determine that the predicted output is reasonable and adjust the output volume.

[0126] Specifically, the trained litchi volume prediction model is used to predict the volume of the data to be predicted. The predicted volumes are then summed to obtain the orchard litchi yield, including:

[0127] Four cameras are used to capture images of lychee trees. Then, the output volumes from step eight are summed to estimate the lychee yield of the tree.

[0128]

[0129] Where y is the volume of the r-th lychee fruit in the image taken from the j-th camera position on the i-th tree, and ρ represents the density of the lychee fruit.

[0130] Aerial images of the lychee orchard were obtained using drones, and the pixel sizes L1,…,L of the lychee trees in step nine were obtained at a certain height h1. m Aerial photography was used to capture h2 of the entire lychee orchard, and the pixel size of the aerial images from step 9 was recorded as K1,…,K. m If the total pixel area is U, then the estimated lychee yield of the entire orchard is:

[0131]

[0132] In this invention, the data to be predicted is input into the litchi volume prediction model. The trained litchi volume prediction model is used to predict the volume of the data to be tested, and the reasonableness of the predicted data is verified. Then, the predicted litchi volume is output.

[0133] The present invention also provides a computer device, comprising:

[0134] Memory, which stores computer programs;

[0135] A processor, which executes the computer program to implement the image vision-based litchi recognition and yield estimation method.

[0136] The signal receiver connects to the camera unit and drone hangar via internal circuitry.

[0137] The present invention also provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the image vision-based litchi recognition and yield estimation method described above.

[0138] Example 1

[0139] A method for litchi identification and yield estimation based on image vision, comprising:

[0140] 1. Acquire various data, such as orchard lychee trees and natural environment, as training data; wherein, the training data includes lychee tree image data and lychee fruit image data.

[0141] 2. Perform corresponding preprocessing on the training data according to the data type: adjust the brightness, perform median filtering, and adjust the contrast of the captured litchi tree images in sequence to obtain the litchi tree outline image.

[0142] 3. Using the preprocessed litchi image training and prediction data, litchi fruit identification is performed based on the Haar-Like features in the matrix features; the AdaBoost algorithm is used to construct a strong classifier by using the matrix features as a weak classifier.

[0143] 31. Because the structure of lychee fruits differs from that of leaves, the sky, and other graphic elements, and rectangular features are more sensitive to simple graphic structures such as line segments and edges, the calculated feature values ​​for lychee fruits should differ from those for leaves, the sky, etc. Therefore, lychee fruit identification is based on Haar-Like features in the matrix. Specifically, a search window is set up, and the feature value is equal to the sum of pixel values ​​in the black areas of the matrix minus the sum of pixel values ​​in the white areas, as shown below. Figure 4 As shown.

[0144] Using an m×m pixel resolution detector, the total number of rectangles within it that satisfy specific conditions can be calculated as follows: For an m×m sub-window, determining the top-left vertex A(x1, y1) and bottom-right vertex B(x2, y2) is sufficient to determine a rectangle; if this rectangle must also satisfy the following two conditions (called the (s,t) conditions, and a rectangle satisfying the (s,t) conditions is called a condition rectangle):

[0145] 1. The length of the side in the x-direction must be divisible by the natural number s (and can be divided into s equal segments);

[0146] 2. The length of the side in the y-direction must be divisible by the natural number t (it must be able to be divided into t equal segments);

[0147] Therefore, the minimum size of this rectangle is s×t or t×s, and the maximum size is... or in This is the floor operator.

[0148] Therefore, the range of values ​​for vertices A and B can be determined, specifically A(x1,y2): x1∈(1,2,…,ms,m-s+1), y1∈(1,2,…,mt,m-t+1); and B(x2,y2): x2∈(x1+s-1,2,…,ms,m-s+1), y1∈(1,2,…,mt,m-t+1).

[0149] In summary, the number of rectangles satisfying the (s,t) condition within the m×m sub-window is:

[0150]

[0151] From the above formula, we can see that the (s,t) condition describes the characteristics of the rectangular feature. The (s,t) conditions corresponding to different rectangular features are listed below, such as... Figure 5 As shown.

[0152] Therefore, the total number of features Ω for all 6 feature templates in the m×m sub-window is... m This is the sum of the number of rectangular features that satisfy each of the six (s,t) conditions, i.e.:

[0153]

[0154] In particular, since feature templates 1 and 2, 3 and 4 have rotational symmetry, they can be further simplified to:

[0155]

[0156] 32. Use the AdaBoost algorithm to construct a strong classifier by using matrix features as a weak classifier. For a given training set of litchi fruits (A, B), if A and B correspond to labeled positive and negative samples respectively, then a1,…a m ∈A, b1,…b m Let B = {-1, 1}. First, perform n random samplings with replacement in the training set, for a total of N rounds, thus obtaining N training sets, each containing n samples.

[0157] Next, according to the probability distribution D of the training samples t (i) = (1,…,m) selects a weak classifier. Initialize the sample weights as follows: The iteration begins at t = 1, ..., T. The iteration steps are shown below.

[0158] For the constructed multiple weak classifiers, calculate their error rates, and then select the weak classifier with the lowest error rate.

[0159] If ε t If the error rate is greater than 0.5, the iteration ends, because the error rate represents the classification accuracy of the weak classifier. If the error rate is greater than 0.5, it means that the classification accuracy is lower than the accuracy of random guessing, and in this case, the construction of the weak classifier should be stopped immediately.

[0160] Calculated weighting coefficients

[0161] Update sample weights

[0162] When updating the sample weights D based on the sample weight coefficients calculated from the error rate, the weights of correctly classified samples should be decreased, and the weights of misclassified samples should be increased (because h...). i (x i )={-1,1},y i={-1,1}). In this way, we can start with the first weak classifier and gradually build classifiers for simple features and then for complex features. Z t Its function is to update the total weight value to 1.

[0163] Each time a training set is used, a weak classifier h is obtained. After N iterations, N weak classifiers are obtained. These N classifiers are superimposed according to the sample weights corresponding to their weight coefficients to form the final strong classifier H. For the problem of lychee fruit identification and judgment, H will select the result of the optimal weak classifier to obtain the lychee identification region.

[0164] 33. Based on the litchi identification region, use sequence values ​​to label litchi fruits in different regions of the image to obtain a label matrix;

[0165] 34. Convert the RGB image to an HSV image, use Gaussian filtering + Canny high threshold edge detection to remove small isolated areas. In HSV, mark the color temperature of the lychee recognition area as 255 for red and 125 for cyan, and mark other non-recognition areas as 0.

[0166] Furthermore, the regionprops algorithm is used to measure the image region attributes, and the area distribution of the marked regions is statistically analyzed to obtain the pixel plane area of ​​ripe lychees and the pixel plane area of ​​unripe lychees.

[0167] 4. Input the data of the identified image into the litchi volume prediction model, and use a multilayer perceptron to derive and determine the connection weights on the unit paths of the litchi volume prediction model.

[0168] 41. Assuming that lychees are ideally symmetrical and their images are elliptical, find the longest side of the 2D image of the lychee based on the pixels. Using the longest side as the X-axis and the center of the X-axis as the Y-axis, apply formula a... 2 =b 2 +c 2 and Substitute the values ​​into the equations to find the values ​​of a, b, and c.

[0169] 42. For identifying the lychee region, perform horizontal and vertical pixel overlay. Based on the pixel accumulation in the horizontal and vertical directions, obtain the number of minimum values ​​d. This value can be used to determine the number of lychees in the image.

[0170] 43. Obtain the pixel values ​​a, b, c and the number of local minima d corresponding to the above lychee image as training input values, and use the volume of the lychee as the training output value. Input data is input from the input layer, passes through intermediate layers, and is finally output from the output layer. An error signal is obtained by comparing the actual output with the expected output. This error signal is propagated forward layer by layer from the output layer to obtain the error signal for each layer. The error is then reduced by adjusting the connection weights of each layer. The weight adjustment mainly uses gradient descent, continuously adjusting the weights to minimize the error, thereby learning the optimal connection weights. Specifically, this includes:

[0171] The data to be predicted is input into the litchi volume prediction model. The trained litchi volume prediction model is then used to predict the litchi volume based on the data to be predicted. The structure of the litchi volume prediction model is as follows:

[0172] To facilitate the derivation, a, b, c, and d are sorted and represented as x1, x2, x3, and x4 respectively, with y as the output. The derivation is performed using a multilayer perceptron. 1ij w represents the connection weights between the input layer and the intermediate layer. 2j1 This represents the connection weights between the intermediate visible layer and the output layer. 'i' represents the input unit, and 'j' represents the intermediate layer unit.

[0173] First, adjust the connection weights between the intermediate and output layers. Adjust the error function E relative to the connection weights w. 2ij Expand the derivative into a composite function and then find its derivative.

[0174]

[0175] set up Find the error function E. The derivative of .

[0176]

[0177] Let g = -x, h = e g If p = 1 + h, then f(u) = p -1 According to the derivation, we can obtain:

[0178]

[0179] After simplification, we get:

[0180]

[0181] And because For w 2j1 The result of the derivative depends only on the value of the intermediate layer, therefore let z j This represents the value of the intermediate layer. In summary, the connection weight adjustment values ​​are as follows:

[0182]

[0183]

[0184] Next, the weights of the connection layers between the input layer and the intermediate layers are adjusted. The adjustment values ​​of the connection weights between the input layer and the intermediate layers are determined based on the error function of the output layer, and the derivative formula is shown below.

[0185]

[0186] Similar to the calculation method for the weight adjustment values ​​between the intermediate and output layers, the output y is also calculated by expanding the formula and then taking the derivative of the composite function. The difference is that here it represents the activation values ​​between the intermediate and output layer units. Connection weights between the output layer and the intermediate layer Differentiate. u 21 The value z of the intermediate layer j and connection weight w 2j1 Calculated. For z j Differentiation yields the following equation:

[0187]

[0188] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z of the intermediate layer j Taking the derivative, the result is only related to the connection weight w. 2j1 Related.

[0189]

[0190] Below is the value z of the intermediate layer. j For connection weights Find the derivative.

[0191]

[0192] Similar to y above, z j It is also the sigmoid function, with respect to z. j Taking the derivative, we get the following equation:

[0193]

[0194] Below is the activation function u between the input layer and the intermediate layer units. 1j The connection weight w between the intermediate layer and the output layer 1ij The derivative is obtained by taking the derivative and the result is only related to the input value x. i Related to, i.e.:

[0195]

[0196] In summary, the derivatives of the constituent units and connection weights have been obtained.

[0197]

[0198] The connection weight w between the input layer and the intermediate layer 1ij Find the derivative.

[0199]

[0200] The final connection weight adjustment values ​​between the intermediate layer and the output layer are shown below:

[0201]

[0202] The adjusted connection weights between the input layer and the intermediate layer are shown below:

[0203]

[0204] Furthermore, the step of performing a correlation test between the predicted lychee volume and the actual volume, and outputting a reasonable prediction result based on the correlation test, includes:

[0205] Determine whether the predicted value is greater than or less than a reasonable threshold;

[0206] If so, the predicted output is deemed unreasonable;

[0207] If not, determine that the predicted output is reasonable and adjust the output volume.

[0208] 5. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and sum the predicted volumes to obtain the litchi yield of the orchard.

[0209] 51. Four cameras are used to capture images of lychee trees. The images are then summed based on the output volumes from step eight. The estimated yield of lychee fruit from the tree is then:

[0210]

[0211] Where y is the volume of the r-th lychee fruit in the image taken from the j-th camera position on the i-th tree, and ρ represents the density of the lychee fruit.

[0212] 52. Use drones to capture aerial images of the lychee orchard, and obtain the pixel sizes L1,…,L of the lychee tree aerial images from step nine at a certain height h1. m Aerial photography was used to capture h2 of the entire lychee orchard, and the pixel size of the aerial images from step 9 was recorded as K1,…,K. m If the total pixel area is U, then the estimated lychee yield of the entire orchard is:

[0213]

[0214] Example 2

[0215] A computer device, comprising:

[0216] Memory, which stores computer programs;

[0217] Figure 3 This is a diagram illustrating the internal structure of a computer device according to one embodiment. For example... Figure 3 As shown, the device includes a computer, a camera, and a drone hangar. The computer includes a processor, memory, a database, and a signal receiver connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the computer program is executed by the processor, it implements the strong classifier and litchi volume prediction model described in the first aspect.

[0218] The components of the memory and processor are electrically connected directly or indirectly to enable data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines. The processor includes at least one software functional module, which can be stored in memory or embedded in the server's operating system (OS) in the form of software or firmware. The processor is used to execute the executable module stored in memory.

[0219] The memory can be Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory is used to store programs and voice data; the processor executes the program after receiving the execution instruction.

[0220] The processor may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an On-Premises Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0221] The processor couples various input / output devices to itself and memory. In some embodiments, the processor and memory can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0222] Peripheral interfaces couple various input / output devices to the processor and memory. In some embodiments, the peripheral interface, processor, and memory can be implemented on a single chip. In other instances, they can be implemented on separate chips.

[0223] The signal receiver connects to the camera unit and the drone hangar via internal circuitry. In this example, it communicates with the drone hangar to convert various control signals for operating the drone into radio frequency (RF) signals, which are then transmitted through an antenna. After propagating through the air, the RF signals are received by the drone's receiving antenna and sent to a receiver for demodulation. Ultimately, this generates control signals, which are then sent to the flight controller or servo unit to complete the shooting task and return the images.

[0224] Example 3

[0225] A computer storage medium storing a computer program is disclosed. The computer storage medium may be a magnetic random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a flash memory, a magnetic surface memory, an optical disc, or a read-only optical disc, etc.; it may also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc. When the computer program is executed by a processor, it implements the strong classifier and litchi volume prediction model described in the first aspect.

[0226] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A method for litchi recognition and yield estimation based on image vision, characterized in that, Includes the following steps: A. Obtain the outline of the lychee trees in the orchard, the image data of the lychee fruits, and the volume data of the lychees as training data; B. Perform corresponding preprocessing on the training data according to the data type; C. Using the preprocessed litchi image training data and prediction data, litchi fruit identification is performed based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to use the matrix features as a weak classifier to construct a strong classifier in order to obtain the litchi identification region. D. Input the data of the identified image into the litchi volume prediction model, and use a multilayer perceptron to derive and determine the connection weights on the unit path of the litchi volume prediction model. D1. Assuming that lychees are ideally symmetrical and their images are elliptical, find the longest side of the two-dimensional image of a lychee based on the pixels. Using the longest side as the X-axis and the center of the X-axis as the Y-axis, rely on formula a... 2 =b 2 +c 2 and Substitute the values ​​into the equations to find the values ​​of a, b, and c respectively. D2. For identifying the lychee region, perform horizontal and vertical pixel overlay. Based on the pixel accumulation in the horizontal and vertical directions, obtain the number of minimum values ​​d. This number of minimum values ​​d can determine the number of lychees in the image. D3. Obtain the values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above lychee image as training input values, and use the volume of the lychee as the training output value. Input data is input from the input layer, passes through the intermediate layer, and is finally output from the output layer. By comparing the actual output and the expected output, the error signal is obtained. The error signal is propagated from the output layer forward layer by layer to obtain the error signal of each layer. Then, the error is reduced by adjusting the connection weights of each layer. E. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and sum the predicted volumes to obtain the litchi yield of the orchard.

2. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step B specifically involves sequentially adjusting the brightness, performing median filtering, and adjusting the contrast of the captured lychee tree image to obtain the lychee tree outline image.

3. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step C involves identifying litchi fruits based on the Haar-Like features in the matrix. Specifically, a search window is set up, and the feature value is equal to the sum of the pixel values ​​in the black area of ​​the matrix minus the sum of the pixel values ​​in the white area. The matrix area is a detector with an m×m pixel resolution. The black and white areas are divided based on four features: edge features, linear features, center features, and diagonal features, to identify litchi fruits.

4. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step C also includes: based on the litchi identification region, using sequence values ​​to label litchi fruits in different regions of the image to obtain a label matrix, and converting the RGB image into an HSV image, and using Gaussian filtering + Canny high-threshold edge detection to remove small isolated regions.

5. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step D3 involves adjusting the weights using gradient descent. The weights are continuously adjusted to minimize the error, thereby learning the optimal connection weights. Specifically: Assuming the litchi image is an ellipsoid, and obtaining the corresponding parameters a, b, c, d, these four parameters are expressed as x1, x2, x3, x4, and y is the output. A multilayer perceptron is used for derivation, and w... 1ij w represents the connection weights between the input layer and the intermediate layer. 2j1 This represents the connection weight between the intermediate visible layer and the output layer, where i represents the input unit and j represents the intermediate layer unit; D31. Adjust the connection weights between the intermediate layer and the output layer, and adjust the error function E relative to the connection weights w. 2ij Differentiate and expand to find the derivative of the composite function; set up Find the error function E. The derivative; Let g = -x, h = e g If p = 1 + h, then f(u) = p -1 According to the derivation, we can obtain: After simplification, we get: And because For w 2j1 The result of the derivative depends only on the value of the intermediate layer; therefore, let z j This represents the value of the intermediate layer. In summary, the connection weight adjustment values ​​are as follows: D32. Adjust the weights of the connection layers between the input layer and the intermediate layers; the adjustment value of the connection weights between the input layer and the intermediate layers is determined based on the error function of the output layer, and the derivative formula is shown below: Similar to the calculation method for the weight adjustment values ​​between the intermediate and output layers, the output y is also calculated by expanding the formula and then taking the derivative of the composite function. However, here it represents the activation values ​​between the intermediate and output layer units. Connection weights between the output layer and the intermediate layer Differentiate, u 21 The value z of the intermediate layer j and connection weight w 2j1 Calculations show that for z j Differentiation yields the following equation: Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z of the intermediate layer j Taking the derivative, the result is only related to the connection weight w. 2j1 Related; Below is the value z of the intermediate layer. j For connection weights Differentiate; Similar to y above, z j It is also the sigmoid function, with respect to z. j Taking the derivative, we get the following equation: Below is the activation function u between the input layer and the intermediate layer units. 1j The connection weight w between the intermediate layer and the output layer 1ij The derivative is obtained by taking the derivative and the result is only related to the input value x. i Relevant, namely: In summary, the derivatives of the constituent units and connection weights have been obtained. The connection weight w between the input layer and the intermediate layer 1ij Differentiate; D33. The connection weight adjustment values ​​between the intermediate layer and the output layer are shown below: The adjusted connection weights between the input layer and the intermediate layer are shown below:

6. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step E involves using the trained lychee volume prediction model to predict the volume of the data to be predicted. Specifically, the predicted lychee volume is compared with the actual volume. If the correlation test is reasonable, the predicted volume is determined to be greater than or less than a reasonable threshold. If so, the predicted volume is determined to be unreasonable. If not, the predicted volume is determined to be reasonable and the volume is output.

7. The method for litchi recognition and yield estimation based on image vision according to claim 1, characterized in that, Step E involves summing the predicted volumes to obtain the orchard's lychee yield, specifically including the following steps: E1. If four cameras are used to capture images of lychee trees, and then the output volumes are summed, the estimated yield of the lychee trees can be calculated as follows: Where y is the volume of the r-th lychee fruit in the image taken from the j-th camera position on the i-th tree, and ρ represents the density of the lychee fruit; E2. Use drones to capture aerial images of the lychee orchard, and obtain the pixel size L1,…,L of the aerial images of the lychee trees at a certain height h1. m Aerial photography was used to capture the entire lychee orchard (h2), and the pixel size of the aerial images was recorded as K1,…,K. m If the total pixel area is U, then the estimated lychee yield of the entire orchard is:

8. A computer device, characterized in that, include: Memory, which stores computer programs; A processor, which, when executing the computer program, implements a method for litchi recognition and yield estimation based on image vision as described in any one of claims 1-7; The signal receiver connects to the camera unit and drone hangar via internal circuitry.

9. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements a method for litchi identification and yield estimation based on image vision as described in any one of claims 1-7.

Images