Litchi identification and yield estimation method and device based on image vision and medium

Abstract

The invention relates to a litchi recognition and yield estimation method and device based on image vision and a medium, and the method comprises the steps: obtaining image data of the contour of a litchi fruit tree in an orchard, the litchi fruit and the like as training data, and carrying out the preprocessing; carrying out litchi fruit identification on the preprocessed training data in Haar-Like features in the matrix features; inputting the data of the identified image into a litchi volume prediction model, and deducing by using a multi-layer sensor to obtain a connection weight; performing volume prediction on to-be-predicted data by using the trained litchi volume prediction model; and adding the predicted volumes to obtain the yield of the orchard litchis. According to the method, image data of the contour of the litchi tree, the litchi fruit and the like in the orchard are obtained as prediction input values, the matrix features are used as a weak classifier by using the AdaBoost algorithm, the strong classifier is constructed, the accurate prediction volume of the litchi is obtained by using the error back propagation algorithm, and a foundation is laid for researching the relationship between the weather, the wind power, the fertilization and the like and the yield.

Description

Technical Field

[0001] The invention belongs to the technical field of crop image recognition and yield estimation, and in particular relates to a litchi recognition and yield estimation method, device and medium based on image vision. Background Art

[0002] Litchi is a tropical and subtropical fruit. Although it is loved by people for its unique flavor and nutritional value, its production and management require a lot of manpower and resources, especially in the identification and yield estimation of litchi. Traditional methods usually rely on manual observation and recording, which is not only time-consuming and labor-intensive, but also limited by manual experience and subjective judgment. Therefore, an efficient and accurate litchi identification and yield estimation method is urgently needed to improve production efficiency and management level.

[0003] Existing litchi recognition methods mostly focus on target detection of litchi fruits, mainly using image data of litchi trees for basic image recognition to provide support for bionic picking of litchi fruits. However, these methods are not deep and comprehensive enough in terms of litchi yield estimation and prediction, and cannot provide a strong basis for litchi picking. How to comprehensively analyze litchi image data, achieve accurate detection and yield estimation of litchi fruits, and establish a complete set of litchi recognition and yield estimation methods and systems are the problems that need to be solved urgently in the current digital era of smart agriculture refined picking. Summary of the invention

[0004] The purpose of the present invention is to provide a litchi identification and yield estimation method, device and medium based on image vision, which solves the problem of comprehensive analysis of litchi image data, realizes accurate detection and yield estimation of litchi fruits, and greatly improves the efficiency and accuracy of litchi planting management by combining computer vision, machine learning and image processing technologies.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A method for identifying and estimating litchi yield based on image vision comprises the following steps:

[0007] A obtains the outline of litchi trees in the orchard, litchi fruit image data and litchi volume data as training data;

[0008] B. Preprocess the training data accordingly according to the data type;

[0009] C. Using the preprocessed litchi image training data and prediction data, the litchi fruit is identified based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to take the matrix features as weak classifiers to construct a strong classifier to obtain the litchi identification area.

[0010] D. inputting the data of the recognized image into the litchi volume prediction model, and using a multi-layer perceptron to derive and determine the connection weights on the unit paths of the litchi volume prediction model;

[0011] E. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and add up the predicted volumes to obtain the litchi yield in the orchard.

[0012] Furthermore, step B specifically includes: adjusting brightness, median filtering, and contrast adjustment on the photographed litchi tree image in sequence to obtain a litchi tree outline image.

[0013] Furthermore, step C is to identify litchi fruits based on Haar-Like features in the matrix features, specifically: a search window is set, the feature value is equal to the sum of the pixel values ​​of the black area in the matrix minus the sum of the pixels of the white area, wherein the matrix area is a detector with an m×m pixel resolution, and the black and white areas are divided into four features based on Haar-Like features, namely edge features, linear features, center features, and diagonal features for litchi fruit identification.

[0014] Furthermore, step C also includes: based on the litchi recognition area, using the sequence value to mark the litchi fruits in different areas of the image to obtain a label matrix, and converting the RGB image into an HSV image, and using Gaussian filtering + Canny high threshold edge detection to remove small isolated areas.

[0015] Further, step D comprises the following steps:

[0016] D1. Assuming that the symmetrical image of litchi is an ellipse in an ideal state, find the longest side of the two-dimensional image of litchi according to the pixels, take the longest side as the X-axis, take the center of the X-axis as the Y-axis, and rely on formula a 2 =b 2 +c 2 and Substitute the numerical values ​​to find the values ​​of a, b, and c respectively;

[0017] D2, for identifying litchi area, horizontal and vertical pixel superposition is performed, and the number of minimum values ​​d is obtained according to the accumulation of horizontal and vertical pixels, which can be used to determine the number of litchi in the image;

[0018] D3. Obtain the values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above litchi image as training input values, and use the volume of litchi as training output values. Input data is input from the input layer, passes through the intermediate layer, and is finally output from the output layer. An error signal is obtained by comparing the actual output with the expected output. The error signal is propagated forward from the output layer layer by layer to obtain error signals of each layer, and then the connection weights of each layer are adjusted to reduce the error.

[0019] Furthermore, in step D3, the weight adjustment uses the gradient descent method to continuously adjust the weight to minimize the error, and learn to obtain the optimal connection weight, which is specifically:

[0020] Assume that the litchi image is an ellipsoid, obtain the corresponding parameters a, b, c, d, and sort the four parameters to represent x1, x2, x3, x4, y as the output, and use the multilayer perceptron to derive w 1ij represents the connection weight between the input layer and the intermediate layer, w 2j1 represents the connection weight between the intermediate visible layer and the output layer, i represents the input unit, and j represents the intermediate layer unit;

[0021] D31, adjust the connection weight between the intermediate layer and the output layer, and convert the error function E to the connection weight w 2ij The derivative is expanded into the derivative of a composite function;

[0022]

[0023] set up Find the error function E The derivative of

[0024]

[0025] Let g = -x, h = e g , p=1+h, then f(u)=p -1 , according to the derivation:

[0026]

[0027] After simplification, we can get:

[0028]

[0029] Also because For w 2j1 The result of the derivation is only related to the value of the middle layer, so let z j It represents the value of the middle layer. In summary, the connection weight adjustment value is as follows:

[0030]

[0031]

[0032] D32, adjust the weight of the connection layer between the input layer and the middle layer; the connection weight adjustment value between the input layer and the middle layer is determined according to the error function of the output layer, and the derivation formula is as follows:

[0033]

[0034] The calculation method of the weight adjustment value between the middle layer and the output layer is the same. The output y is also the derivative of the composite function after the formula is expanded, but here it is the activation value between the middle layer and the output layer unit The connection weights between the output layer and the intermediate layer Derivation, u 21 The value z in the middle layer j and connection weight w 2j1 Calculated, for z j The following formula can be obtained by derivation:

[0035]

[0036] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z in the middle layer j Derivation, the result is only related to the connection weight w 2j1 Related;

[0037]

[0038] Below is the value z of the middle layer j Connection weight Derivation;

[0039]

[0040] Same as y above, z j It is also a sigmoid function, for z j Taking the derivative, we get the following formula:

[0041]

[0042] Below is the activation function u between the input layer and the intermediate layer units 1j The connection weight w between the intermediate layer and the output layer 1ij Derivative, the result is only equal to the input value x i Relevant, namely:

[0043]

[0044] In summary, the derivation of all units and connection weights is completed;

[0045]

[0046] For the connection weight w between the input layer and the intermediate layer 1ij Derivation;

[0047]

[0048] D33, the connection weight adjustment value between the intermediate layer and the output layer is as follows:

[0049]

[0050] The obtained connection weight adjustment values ​​between the input layer and the intermediate layer are as follows:

[0051]

[0052] Furthermore, in step E, the trained litchi volume prediction model is used to perform volume prediction on the data to be predicted, specifically: a correlation test is performed on the litchi prediction result and the true volume, and the correlation test is output as a reasonable prediction result, that is, whether the prediction value is greater than or less than a reasonable threshold value; if so, the prediction output is judged to be unreasonable; if not, the prediction output is judged to be reasonable and the volume is output.

[0053] Further, step E, adding the predicted volumes to obtain the litchi yield in the orchard, specifically comprises the following steps:

[0054] E1. Four cameras are used to shoot the litchi image of a tree. Then, the output volume is summed up. The estimated litchi yield of a tree is:

[0055]

[0056] Where y is the volume of the rth litchi fruit in the image taken by the jth camera position of the i-th tree, and ρ represents the density of litchi fruit;

[0057] E2. Use drones to obtain aerial images of litchi orchards, and obtain aerial images of litchi trees at a certain height h1 with pixel sizes L1,…,L m , and obtain the h2 of the entire litchi orchard by aerial photography, and record the pixel size of the aerial image as K1,…,K m , the pixel area of ​​the entire region is U, then the estimated litchi yield of the entire orchard is:

[0058]

[0059] A computer device comprising:

[0060] a memory storing a computer program;

[0061] A processor, wherein when executing the computer program, the processor implements the method for identifying and estimating the yield of litchi based on image vision;

[0062] Signal receiver,The signal receiver is connected with the camera unit and the drone library through internal circuits.

[0063] A computer storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method for identifying and estimating the yield of litchi based on image vision is implemented.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] The present invention obtains image data such as the outline of litchi trees and litchi fruits in an orchard as prediction input values, uses the AdaBoost algorithm to take matrix features as weak classifiers, constructs a strong classifier, and uses the error back propagation algorithm to obtain accurate predicted volume of litchi, laying the foundation for studying the relationship between weather, wind power, fertilization, etc. and yield, and providing a reference basis for smart agricultural planting; by combining computer vision, machine learning, image processing and other technologies, the efficiency and accuracy of litchi planting management are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0067] Figure 1 A flowchart of the steps of the litchi identification and yield estimation method based on image vision of the present invention;

[0068] Figure 2 Schematic diagram of the error back propagation algorithm;

[0069] Figure 3 A diagram of the internal structure of a computer device;

[0070] Figure 4 Schematic diagram of litchi fruit recognition based on Haar-Like features in matrix features;

[0071] Figure 5 (s, t) conditions corresponding to different rectangular features;

[0072] Figure 6a Based on the litchi recognition area, the sequence values ​​are used to mark the litchi fruits in different areas of the image to obtain a label matrix;

[0073] Figure 6b Depend on Figure 6a Make a coordinate system marking diagram;

[0074] Figure 6c Mathematical idealized model. DETAILED DESCRIPTION

[0075] The present invention will be further described below in conjunction with embodiments:

[0076] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0077] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0078] The present invention provides a method for identifying and estimating litchi yield based on image vision, comprising the following steps:

[0079] 1. Obtain image data such as the outline of litchi trees in the orchard, litchi fruits, and litchi volume data as training data.

[0080] 2. Perform corresponding preprocessing on the training data according to the data type.

[0081] 3. Using the preprocessed litchi image training data and prediction data, litchi fruit recognition is performed based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to use the matrix features as weak classifiers to build a strong classifier.

[0082] Specifically, using Haar-Like features to identify litchi fruits includes the following steps: setting a search window, the eigenvalue is equal to the sum of the pixel values ​​of the black area in the matrix minus the sum of the pixels of the white area; and the matrix area is to set a detector with an m×m pixel resolution, and the division of black and white areas is based on Haar-Like features: edge features, linear features, center features, and diagonal features are divided into four features for litchi fruit recognition.

[0083] It also includes: based on the litchi recognition area, using sequence values ​​to mark the litchi fruits in different areas of the image to obtain a label matrix, and converting the RGB image into an HSV image, using Gaussian filtering + Canny high threshold edge detection to remove small isolated areas.

[0084] 4. Input the data of the recognized image into the litchi volume prediction model, use the multi-layer perceptron to deduce, and determine the connection weights on the unit path of the litchi volume prediction model, including the following steps:

[0085] 41. Assuming that the symmetrical image of litchi is an ellipse in an ideal state, find the longest side of the two-dimensional image of litchi according to the pixels, take the longest side as the X-axis, take the center of the X-axis as the Y-axis, and rely on formula a 2 =b 2 +c 2 and Substitute the numerical values ​​to find the values ​​of a, b, and c respectively;

[0086] 42. For identifying the litchi area, perform horizontal and vertical pixel superposition, and obtain the number of minimum values ​​d according to the accumulation of horizontal and vertical pixels. This value can be used to determine the number of litchi in the image;

[0087] 43. Error back propagation algorithm: The values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above litchi image are obtained as training input values, and the volume of litchi is used as the training output value. The input data is input from the input layer, passes through the middle layer, and is finally output from the output layer. The error signal is obtained by comparing the actual output with the expected output. The error signal is propagated forward from the output layer layer by layer to obtain the error signal of each layer, and then the connection weights of each layer are adjusted to reduce the error.

[0088] Among them, the weight adjustment uses the gradient descent method to continuously adjust the weight to minimize the error, and learn to obtain the optimal connection weight, specifically:

[0089] The litchi image is idealized as an ellipsoid to obtain the corresponding parameters a, b, c, d. The four parameters are sorted and expressed as x1, x2, x3, x4, and y is the output. Multilayer perceptron is used to derive w 1ij represents the connection weight between the input layer and the intermediate layer, w 2j1 Represents the connection weight between the intermediate visible layer and the output layer. i represents the input unit and j represents the intermediate layer unit.

[0090] First, let’s adjust the connection weights between the middle layer and the output layer. 2ij The derivative is expanded into the derivative of a composite function.

[0091]

[0092] set up Find the error function E The derivative of .

[0093]

[0094] Let g = -x, h = e g , p=1+h, then f(u)=p -1 . According to the derivation, we can get:

[0095]

[0096] After simplification, we can get:

[0097]

[0098] Also because For w 2j1 The result of the derivation is only related to the value of the middle layer, so let z j It represents the value of the middle layer. In summary, the connection weight adjustment value is as follows:

[0099]

[0100]

[0101] Next, adjust the weight of the connection layer between the input layer and the middle layer. The connection weight adjustment value between the input layer and the middle layer is determined based on the error function of the output layer, and the derivation formula is as follows.

[0102]

[0103] The calculation method of the weight adjustment value between the middle layer and the output layer is the same. The output y is also the derivative of the composite function after the formula is expanded. The difference from the previous one is that here is the activation value between the middle layer and the output layer unit The connection weights between the output layer and the intermediate layer Derivative. 21 The value z in the middle layer j and connection weight w 2j1 Calculate. j The following formula can be obtained by derivation:

[0104]

[0105] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z in the middle layer j Derivation, the result is only related to the connection weight w 2j1 Related.

[0106]

[0107] Below is the value z of the middle layer j Connection weight Derivative.

[0108]

[0109] Same as y above, z j It is also a sigmoid function, for z j Taking the derivative, we get the following formula:

[0110]

[0111] Below is the activation function u between the input layer and the intermediate layer units 1j The connection weight w between the intermediate layer and the output layer 1ij Derivative, the result is only equal to the input value x i Relevant, namely:

[0112]

[0113] In summary, the derivation of all units and connection weights is completed.

[0114]

[0115] For the connection weight w between the input layer and the intermediate layer 1ij Derivative.

[0116]

[0117] Finally, the connection weight adjustment values ​​between the intermediate layer and the output layer are as follows:

[0118]

[0119] The obtained connection weight adjustment values ​​between the input layer and the intermediate layer are as follows:

[0120]

[0121] 5. Use the trained litchi volume prediction model to predict the volume of the data to be predicted; add up the predicted volumes to obtain the litchi yield in the orchard.

[0122] The trained litchi volume prediction model is used to predict the volume of the data to be predicted. Specifically, the litchi prediction result is tested for correlation with the actual volume, and the correlation test is output as a reasonable prediction result, including:

[0123] Determine whether the predicted value is greater than or less than a reasonable threshold;

[0124] If so, the prediction output is judged to be unreasonable;

[0125] If not, the predicted output is judged to be reasonable and the volume is output.

[0126] Specifically, the trained litchi volume prediction model is used to predict the volume of the data to be predicted, and the predicted volumes are added to obtain the litchi yield of the orchard, including:

[0127] Four cameras are used to shoot the litchi image of a tree. Then, the output volume of step 8 is summed up. The estimated litchi yield of a tree is:

[0128]

[0129] Where y is the volume of the rth litchi fruit in the image taken by the jth camera position of the i-th tree, and ρ represents the density of litchi fruits.

[0130] Use drones to obtain aerial images of litchi orchards, and obtain aerial images of litchi trees in step nine at a certain height h1 with pixel sizes L1,…,L m , and obtain the h2 of the entire litchi orchard by aerial photography, and record the pixel size of the aerial image in step 9 as K1,…,K m , the pixel area of ​​the entire region is U, then the estimated litchi yield of the entire orchard is:

[0131]

[0132] In the present invention, the data to be predicted is input into the litchi volume prediction model, the trained litchi volume prediction model is used to predict the volume of the data to be tested, and the rationality of the predicted data is checked and judged, and then the predicted litchi volume is output.

[0133] The present invention also provides a computer device, comprising:

[0134] a memory storing a computer program;

[0135] A processor, wherein when executing the computer program, the processor implements the image-based vision-based litchi identification and yield estimation method;

[0136] Signal receiver,The signal receiver is connected with the camera unit and the drone library through internal circuits.

[0137] The present invention also provides a computer storage medium on which a computer program is stored. When the computer program is executed by a processor, the image-based vision-based litchi identification and yield estimation method is implemented.

[0138] Example 1

[0139] A method for identifying and estimating litchi yield based on image vision, comprising:

[0140] 1. Obtain various data such as litchi trees in an orchard and natural environment as training data; wherein the training data includes litchi tree picture data and litchi fruit image data.

[0141] 2. Perform corresponding preprocessing on the training data according to the data type: adjust the brightness, median filter and contrast of the photographed litchi tree image in turn to obtain the litchi tree outline image.

[0142] 3. Using the preprocessed litchi image training data and prediction data, litchi fruit recognition is performed based on the Haar-Like features in the matrix features; the AdaBoost algorithm is used to use the matrix features as weak classifiers to build a strong classifier.

[0143] 31. Because the structure of litchi fruit is different from that of leaves, sky and other graphics, and the rectangular feature is more sensitive to some simple graphic structures such as line segments and edges, the calculated feature value of litchi fruit should be different from that of leaves, sky and other features. Therefore, litchi fruit recognition is performed based on the Haar-Like feature in the matrix feature. Specifically, a search window is set, and the feature value is equal to the sum of the pixel values ​​of the black area in the matrix minus the sum of the pixels of the white area. Figure 4 shown.

[0144] Using a detector with an m×m pixel resolution, the total number of all rectangles that meet certain conditions can be calculated as follows: For an m×m subwindow, a rectangle can be determined by only determining the upper left vertex A(x1, y1) and the lower right vertex B(x2, y2) of the rectangle; if the rectangle must also meet the following two conditions (called the (s, t) condition, a rectangle that meets the (s, t) condition is called a conditional rectangle):

[0145] 1. The length of the side in the x direction must be divisible by a natural number s (can be equally divided into s segments);

[0146] 2. The length of the side in the y direction must be divisible by the natural number t (can be equally divided into t segments);

[0147] Then, the minimum size of this rectangle is s×t or t×s, and the maximum size is or in is the floor operator.

[0148] Therefore, we can determine the value range of vertices A and B, and determine A(x1,y2): x1∈(1,2,…,ms,m-s+1), y1∈(1,2,…,mt,m-t+1); determine B(x2,y2): x2∈(x1+s-1,2,…,ms,m-s+1), y1∈(1,2,…,mt,m-t+1);

[0149] From the above analysis, we can conclude that in the m×m subwindow, the number of all rectangles that satisfy the (s, t) condition is:

[0150]

[0151] From the above formula, we can see that the (s, t) condition describes the characteristics of the rectangular feature, and the (s, t) conditions corresponding to different rectangular features are listed, such as Figure 5 shown.

[0152] Therefore, the total number of features of all 6 feature templates in the m×m subwindow is Ω m , is the sum of the number of rectangular features that satisfy the six (s, t) conditions, namely:

[0153]

[0154] In particular, since feature templates 1 and 2, 3 and 4 have rotational symmetry, they can be further simplified as follows:

[0155]

[0156] 32. Use the AdaBoost algorithm to use the matrix features as weak classifiers to build a strong classifier. For a given litchi fruit training set (A, B), if A and B correspond to labeled positive and negative samples respectively, then a1,…a m ∈A,b1,…b m ∈B={-1,1}. First, perform n random sampling with replacement in the training set, for a total of N rounds, so that N training sets can be obtained, each of which contains n samples.

[0157] Next, according to the probability distribution D of the training samples t (i)=(1,…,m) selects weak classifiers. Initialize sample weights to Iteration starts from t=1…, T. The iteration steps are as follows.

[0158] For the multiple weak classifiers constructed, calculate the error rate of the weak classifiers, and then select the weak classifier with the smallest error rate

[0159] If ε t >0.5, the iteration ends, because the error rate indicates the classification accuracy of the classifier. If the error rate is greater than 0.5, it means that the classification accuracy is lower than the accuracy of random guessing. At this time, it is necessary to stop the construction of the weak classifier immediately;

[0160] Calculated weight coefficient

[0161] Update sample weights

[0162] When updating the sample weight D based on the sample weight coefficient calculated according to the error rate, the weight of the correctly classified samples should be reduced, and the weight of the incorrectly classified samples should be increased (because h i (x i )={-1,1},y i={-1,1}). In this way, starting from the first weak classifier, we can gradually build classifiers for simple features and classifiers for complex features. t The role of is to update the total value of the weight to 1.

[0163] Each time a training set is used, a weak classifier h is obtained. After N cycles, N weak classifiers are obtained. By superimposing the sample weights of these N classifiers according to the weight coefficients, the final strong classifier H will be formed. For the problem of litchi fruit recognition and judgment, H will select the result of the optimal weak classifier to obtain the litchi recognition area.

[0164] 33. Based on the litchi recognition area, use the sequence value to mark the litchi fruits in different areas of the image to obtain a label matrix;

[0165] 34. Convert the RGB image into an HSV image, use Gaussian filtering + Canny high threshold edge detection to remove small isolated areas, and mark the litchi recognition area with a red color temperature of 255 and a cyan color temperature of 125 under HSV, and mark other non-recognition areas with 0.

[0166] Furthermore, the regionprops algorithm is used to measure the function of image region attributes, and the area distribution of the marked regions is statistically analyzed to obtain the pixel plane area of ​​the ripe lychees and the pixel plane area of ​​the lychees to be ripened.

[0167] 4. Input the data of the recognized image into the litchi volume prediction model, use the multi-layer perceptron to deduce, and determine the connection weights on the unit path of the litchi volume prediction model.

[0168] 41. Assuming that the symmetrical image of litchi is an ellipse in an ideal state, find the longest side of the two-dimensional image of litchi according to the pixels, take the longest side as the X-axis, take the center of the X-axis as the Y-axis, and rely on formula a 2 =b 2 +c 2 and Substitute the numerical values ​​and find the values ​​of a, b, and c respectively.

[0169] 42. In order to identify the litchi area, horizontal and vertical pixel superposition is performed. According to the accumulation of horizontal and vertical pixels, the number of minimum values ​​d is obtained. This value can be used to determine the number of litchi in the image.

[0170] 43. Obtain the values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above litchi image as training input values, and use the volume of litchi as training output value. Input data is input from the input layer, passes through the middle layer, and is finally output from the output layer. The error signal is obtained by comparing the actual output with the expected output. The error signal is propagated forward from the output layer layer by layer to obtain the error signal of each layer, and then the connection weights of each layer are adjusted to reduce the error. The weight adjustment mainly uses the gradient descent method, which continuously adjusts the weight to minimize the error, and learns to obtain the optimal connection weight. Specifically including:

[0171] The data to be predicted is input into the litchi volume prediction model, and the trained litchi volume prediction model is used to predict the litchi volume of the data to be predicted. The structure of the litchi volume prediction model is as follows:

[0172] In order to facilitate the derivation, a, b, c, d are sorted and expressed as x1, x2, x3, x4, y is the output, and the multi-layer perceptron is used for derivation. 1ij represents the connection weight between the input layer and the intermediate layer, w 2j1 Represents the connection weight between the intermediate visible layer and the output layer. i represents the input unit and j represents the intermediate layer unit.

[0173] First, let’s adjust the connection weights between the middle layer and the output layer. 2ij The derivative is expanded into the derivative of a composite function.

[0174]

[0175] set up Find the error function E The derivative of .

[0176]

[0177] Let g = -x, h = e g , p=1+h, then f(u)=p -1 . According to the derivation, we can get:

[0178]

[0179] After simplification, we can get:

[0180]

[0181] Also because For w 2j1 The result of the derivation is only related to the value of the middle layer, so let z j It represents the value of the middle layer. In summary, the connection weight adjustment value is as follows:

[0182]

[0183]

[0184] Next, adjust the weight of the connection layer between the input layer and the middle layer. The connection weight adjustment value between the input layer and the middle layer is determined based on the error function of the output layer, and the derivation formula is as follows.

[0185]

[0186] The calculation method of the weight adjustment value between the middle layer and the output layer is the same. The output y is also the derivative of the composite function after the formula is expanded. The difference from the previous one is that here is the activation value between the middle layer and the output layer unit The connection weights between the output layer and the intermediate layer Derivative. 21 The value z in the middle layer j and connection weight w 2j1 Calculate. j The following formula can be obtained by derivation:

[0187]

[0188] Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z in the middle layer j Derivation, the result is only related to the connection weight w 2j1 Related.

[0189]

[0190] Below is the value z of the middle layer j Connection weight Derivative.

[0191]

[0192] Same as y above, z j It is also a sigmoid function, for z j Taking the derivative, we get the following formula:

[0193]

[0194] Below is the activation function u between the input layer and the intermediate layer units 1j The connection weight w between the intermediate layer and the output layer 1ij Derivative, the result is only equal to the input value x i Related to:

[0195]

[0196] In summary, the derivation of all units and connection weights is completed.

[0197]

[0198] For the connection weight w between the input layer and the intermediate layer 1ij Derivative.

[0199]

[0200] The final connection weight adjustment values ​​between the intermediate layer and the output layer are as follows:

[0201]

[0202] The obtained connection weight adjustment values ​​between the input layer and the intermediate layer are as follows:

[0203]

[0204] Furthermore, the correlation test between the litchi prediction result and the actual volume and outputting the correlation test as a reasonable prediction result include:

[0205] Determine whether the predicted value is greater than or less than a reasonable threshold;

[0206] If so, the prediction output is judged to be unreasonable;

[0207] If not, the predicted output is judged to be reasonable and the volume is output.

[0208] 5. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and add the predicted volumes to obtain the litchi yield in the orchard.

[0209] 51. The litchi image of a tree is shot with four cameras, and then the output volume of step 8 is added up. The estimated litchi yield of a tree is:

[0210]

[0211] Where y is the volume of the rth litchi fruit in the image taken by the jth camera position of the i-th tree, and ρ represents the density of litchi fruits.

[0212] 52. Use drones to obtain aerial images of litchi orchards. At a certain height h1, obtain the aerial images of litchi trees in step nine with pixel sizes L1,…,L m , and obtain the h2 of the entire litchi orchard by aerial photography, and record the pixel size of the aerial image in step 9 as K1,…,K m , the pixel area of ​​the entire region is U, then the estimated litchi yield of the entire orchard is:

[0213]

[0214] Example 2

[0215] A computer device comprising:

[0216] a memory storing a computer program;

[0217] Figure 3 FIG. 1 is a diagram showing the internal structure of a computer device according to an embodiment. Figure 3 As shown, the device includes a computer device, a camera, and a drone library, wherein the computer device includes a processor, a memory, a database, and a signal receiver connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the strong classifier and the litchi volume prediction model described in the first aspect are implemented.

[0218] The memory and the processor components are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The processor includes at least one software function module that can be stored in the memory in the form of software or firmware or solidified in the operating system (OS) of the server. The processor is used to execute the executable module stored in the memory.

[0219] The memory may be a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory is used to store programs and voice data, and the processor executes the program after receiving the execution instruction.

[0220] The processor may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0221] The processor couples various input / output devices to the processor and the memory. In some embodiments, the processor and the memory can be implemented in a single chip. In some other examples, they can be implemented by independent chips respectively.

[0222] The peripheral interface couples various input / output devices to the processor and the memory. In some embodiments, the peripheral interface, the processor and the memory can be implemented in a single chip. In some other examples, they can be implemented by separate chips respectively.

[0223] The signal receiver is connected to the camera unit and the drone library through the internal circuit. In the example, the various control signals for controlling the drone are converted into radio frequency signals by connecting with the drone library and transmitted through the antenna; after the radio frequency signal is transmitted through the air, it will be transmitted to the receiver by the receiving antenna on the drone for demodulation, and finally a control signal will be generated and sent to the flight controller or servo unit, finally completing the shooting task and feeding back the image.

[0224] Example 3

[0225] A computer storage medium, wherein a computer program is stored in the computer storage medium, and the computer storage medium may be a magnetic random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a flash memory, a magnetic surface memory, an optical disk, or a read-only optical disk, etc.; or may be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program is executed by a processor, the strong classifier and the litchi volume prediction model described in the first aspect are implemented.

[0226] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for identifying and estimating litchi yield based on image vision, characterized in that: The following steps are involved: A obtains the outline of litchi trees in the orchard, litchi fruit image data and litchi volume data as training data; B. Preprocess the training data accordingly according to the data type; C. Using the preprocessed litchi image training data and prediction data, the litchi fruit is identified based on the Haar-Like features in the matrix features. The AdaBoost algorithm is used to take the matrix features as weak classifiers to construct a strong classifier to obtain the litchi identification area. D. inputting the data of the recognized image into the litchi volume prediction model, and using a multi-layer perceptron to derive and determine the connection weights on the unit paths of the litchi volume prediction model; E. Use the trained litchi volume prediction model to predict the volume of the data to be predicted, and add up the predicted volumes to obtain the litchi yield in the orchard.

2. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step B specifically comprises: adjusting brightness, median filtering, and contrast adjustment in sequence on the photographed litchi tree image to obtain a litchi tree outline image.

3. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step C, based on the Haar-Like features in the matrix features, to identify litchi fruits, specifically: set a search window, the feature value is equal to the sum of the pixel values ​​of the black area in the matrix minus the sum of the pixels of the white area, wherein the matrix area is a detector with an m×m pixel resolution, and the black and white areas are divided into four features based on Haar-Like features, namely, edge features, linear features, center features, and diagonal features, to identify litchi fruits.

4. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step C also includes: based on the litchi recognition area, using sequence values ​​to mark litchi fruits in different areas of the image to obtain a label matrix, and converting the RGB image into an HSV image, and using Gaussian filtering + Canny high threshold edge detection to remove small isolated areas.

5. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step D comprises the following steps: D1. Assuming that the symmetrical image of litchi is an ellipse in an ideal state, find the longest side of the two-dimensional image of litchi according to the pixels, take the longest side as the X-axis, take the center of the X-axis as the Y-axis, and rely on formula a 2 =b 2 +c 2 and Substitute the numerical values ​​to find the values ​​of a, b, and c respectively; D2, for identifying litchi area, horizontal and vertical pixel superposition is performed, and the number of minimum values ​​d is obtained according to the accumulation of horizontal and vertical pixels, which can be used to determine the number of litchi in the image; D3. Obtain the values ​​of pixels a, b, c and the number of minimum values ​​d corresponding to the above litchi image as training input values, and use the volume of litchi as training output values. Input data is input from the input layer, passes through the intermediate layer, and is finally output from the output layer. An error signal is obtained by comparing the actual output with the expected output. The error signal is propagated forward from the output layer layer by layer to obtain error signals of each layer, and then the connection weights of each layer are adjusted to reduce the error.

6. The method for identifying and estimating litchi yield based on image vision according to claim 5, characterized in that: Step D3: The weights are adjusted using the gradient descent method to continuously adjust the weights to minimize the error, and learn to obtain the optimal connection weights, specifically: Assume that the litchi image is an ellipsoid, obtain the corresponding parameters a, b, c, d, and sort the four parameters to represent x1, x2, x3, x4, y as the output, and use the multilayer perceptron to derive w 1ij represents the connection weight between the input layer and the intermediate layer, w 2j1 represents the connection weight between the intermediate visible layer and the output layer, i represents the input unit, and j represents the intermediate layer unit; D31, adjust the connection weight between the intermediate layer and the output layer, and convert the error function E to the connection weight w 2ij The derivative is expanded into the derivative of a composite function; set up Find the error function E The derivative of Let g = -x, h = e g , p=1+h, then f(u)=p -1 , according to the derivation: After simplification, we can get: Also because to w 2j1 The result of the derivation is only related to the value of the middle layer, so let z j It represents the value of the middle layer. In summary, the connection weight adjustment value is as follows: D32, adjust the weight of the connection layer between the input layer and the middle layer; the connection weight adjustment value between the input layer and the middle layer is determined according to the error function of the output layer, and the derivation formula is as follows: The calculation method of the weight adjustment value between the middle layer and the output layer is the same. The output y is also the derivative of the composite function after the formula is expanded, but here it is the activation value between the middle layer and the output layer unit The connection weights between the output layer and the intermediate layer Derivation, u 21 The value z in the middle layer j and connection weight w 2j1 Calculated, for z j The following formula can be obtained by derivation: Among them, the activation value u between the intermediate layer and the output layer unit 21 For the value z in the middle layer j Derivation, the result is only related to the connection weight w 2j1 Related; Below is the value z of the middle layer j Connection weight Derivation; Same as y above, z j It is also a sigmoid function, for z j Taking the derivative, we get the following formula: Below is the activation function u between the input layer and the intermediate layer units 1j The connection weight w between the intermediate layer and the output layer 1ij Derivative, the result is only related to the input value x i Relevant, namely: In summary, the derivation of all units and connection weights is completed; For the connection weight w between the input layer and the intermediate layer 1ij Derivation; D33, the connection weight adjustment value between the intermediate layer and the output layer is as follows: The obtained connection weight adjustment values ​​between the input layer and the intermediate layer are as follows:

7. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step E, using the trained litchi volume prediction model to predict the volume of the data to be predicted, specifically: performing a correlation test on the litchi prediction result and the actual volume, and outputting a reasonable prediction result for the correlation test, that is, judging whether the predicted value is greater than or less than a reasonable threshold; if so, judging the predicted output to be unreasonable; if not, judging the predicted output to be reasonable and outputting the volume.

8. The method for identifying and estimating litchi yield based on image vision according to claim 1, characterized in that: Step E, adding the predicted volumes to obtain the litchi yield in the orchard, specifically comprises the following steps: E1. Four cameras are used to shoot the litchi image of a tree. Then, the output volume is summed up. The estimated litchi yield of a tree is: Where y is the volume of the rth litchi fruit in the image taken by the jth camera position of the i-th tree, and ρ represents the density of litchi fruit; E2. Use drones to obtain aerial images of litchi orchards, and obtain aerial images of litchi trees at a certain height h1 with pixel sizes L1,…,L m , and obtain the h2 of the entire litchi orchard by aerial photography, and record the pixel size of the aerial image as K1,…,K m , the pixel area of ​​the entire region is U, then the estimated litchi yield of the entire orchard is:

9. A computer device, characterized in that: include: a memory storing a computer program; A processor, wherein when executing the computer program, the processor implements the litchi recognition and yield estimation method based on image vision according to any one of claims 1 to 8; Signal receiver,The signal receiver is connected with the camera unit and the drone library through internal circuits.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying and estimating lychees based on image vision as described in any one of claims 1 to 8 is implemented.

Images