A method for locating tea bud picking points based on deep learning algorithm

Abstract

The present invention discloses a method for locating tea bud picking points based on a deep learning algorithm, comprising: firstly, using an image acquisition device to obtain a plurality of tea bud image pairs of tea tree buds, and labeling each pair of tea bud image pairs according to the format of semantic segmentation and key point detection task input data, respectively, to obtain a tea bud semantic segmentation database and a tea bud key point detection database; secondly, the bud semantic segmentation database and the key point detection database are respectively input into a semantic segmentation model and a key point detection model based on deep learning for training, and a semantic segmentation model and a key point detection model are respectively obtained; finally, the trained semantic segmentation model and the key point detection model are used to sequentially process the tea tree bud image pairs to obtain the bud key point positions, and the tea tree bud picking point positions are obtained in combination with the bud growth characteristics. The present invention improves the accuracy and efficiency of tea bud picking point positioning.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence recognition, machine vision and image processing technology, and in particular to a method for locating tea bud picking points based on a deep learning algorithm. Background Art

[0002] Tea is picked mainly by hand and by machine. Hand picking is selective and has high tea quality. People can judge whether the leaf buds are suitable for picking and their grade by observing the shape, color and other characteristics of the leaf buds, and pick the young tea buds at the designated position by breaking them. This method ensures the integrity of the bud shape and has high quality, but it is costly. Due to the optimization and adjustment of the industrial structure and the transfer of labor, there is a labor shortage every year during the tea picking season.

[0003] In recent years, vision-based automatic picking robots have been used to pick premium teas, and the automatic identification and positioning of picking points has become a key point and difficulty that restricts its development. Premium teas are light in weight, and wind or movement of the picking machine will cause the tea leaves to sway; the tea garden environment is complex, and the tea leaves block each other; the light is too strong or too dark, and the resolution of tender buds and old leaves is low. These factors make it very difficult to identify and locate tender bud picking points, which seriously limits the automated picking of premium teas. The current positioning method for picking tea buds has low positioning accuracy and low efficiency. In order to achieve rapid identification and positioning of picking points and ensure the efficiency and high quality requirements of premium tea machine picking, it is necessary to develop a method for obtaining the location information of the picking points of premium teas. Summary of the invention

[0004] The main purpose of the present invention is to propose a tea bud picking point positioning method based on a deep learning algorithm, aiming to improve the accuracy and efficiency of tea bud picking point positioning.

[0005] To achieve the above object, the present invention provides a method for locating tea bud picking points based on a deep learning algorithm, and the method for locating tea bud picking points based on a deep learning algorithm comprises the following steps:

[0006] Acquire a plurality of tea bud image pairs of tea tree buds, wherein the tea bud image pairs include a tea bud thermal image and a tea bud RGB image;

[0007] Labeling each pair of tea bud images to obtain a tea bud semantic segmentation database and a tea bud key point detection database;

[0008] Inputting the sprout semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a trained semantic segmentation model;

[0009] Inputting the sprout key point detection database into the key point detection model to obtain a trained key point detection model;

[0010] Using an image acquisition device to capture an image pair of young tea buds to be picked;

[0011] According to the trained semantic segmentation model, key point detection model and image pairs, the key point positions of the sprouts are obtained;

[0012] The tea tree buds are positioned at picking points according to the bud growth characteristics and the positions of key points of the buds to obtain the positions of the bud picking points.

[0013] Optionally, the step of labeling each pair of tea bud images to obtain a tea bud semantic segmentation database and a tea bud key point detection database comprises:

[0014] Annotating each pair of the tea bud images to obtain semantic segmentation annotation data and key point annotation data corresponding to the tea bud image pair;

[0015] The semantic segmentation annotation data and the key point annotation data are augmented respectively to obtain a tea bud semantic segmentation database and a tea bud key point detection database.

[0016] Optionally, the step of inputting the sprout semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a semantic segmentation model comprises:

[0017] Dividing the sprout semantic segmentation database into a semantic segmentation training set and a semantic segmentation verification set according to a first preset ratio;

[0018] Inputting the semantic segmentation training set into a semantic segmentation model based on deep learning for training, and obtaining a semantic segmentation model weight file;

[0019] Load the trained key point detection model weight file into the key point detection model to obtain the trained semantic segmentation model.

[0020] Optionally, the step of inputting the sprout key point detection database into the key point detection model to obtain the key point detection model includes:

[0021] Dividing the sprout key point detection database into a key point training set and a key point verification set according to a second preset ratio;

[0022] Inputting the key point training set into the key point detection model for training to obtain a key point detection model weight file;

[0023] The trained key point detection model weight file is loaded into the key point detection model to obtain the trained key point detection model.

[0024] Optionally, the step of inputting the key point training set into a key point detection model for training to obtain a tea key point detection model weight file includes:

[0025] A key point detection model is established based on the HRNet network using the pytorch program. The improved key point detection model is to improve the input layer of the key point detection model to 4 channels for inputting the image after the thermal image and the RGB image are fused;

[0026] The key point training set is input into the key point detection model in batches and continuously and iteratively. When the model converges, the training of the key point detection model is completed, and the key point detection model weight file is obtained.

[0027] Optionally, the loss function of the key point detection model is:

[0028] min L=λ1*L1+λ2*L2+λ3*L3;

[0029]

[0030] L2=d(P1,C);

[0031] L3=d(P2,C);

[0032] Among them, λ1, λ2, λ3 are the weight coefficients of L1, L2, L3; Pk represents the kth key point of the sample, Represents the key point heat map predicted by the network, y(P k ) represents the heat map obtained from the true value; d(P1,C) represents the Euclidean distance from the key point of the tea bud predicted by the network to the center C of the rectangular area where it is located.

[0033] Optionally, the step of obtaining the key point positions of the sprouts according to the trained semantic segmentation model, the key point detection model and the image pair comprises:

[0034] The image pairs are semantically segmented using the trained semantic segmentation model to obtain the tea bud detection results;

[0035] The trained key point detection model is used to perform key point detection on the tea bud segmentation results to obtain the key point positions.

[0036] Optionally, the step of locating the tea tree bud picking point based on the bud growth characteristics and the bud key point positions to obtain the bud picking point positions includes:

[0037] When there are two key points of the tender bud, d is the Euclidean distance between the two key points. P1 and P2 are used to establish a straight line equation, and a point 0.4d away from point P1 is taken on the line segment P1P2 as the position of the tea bud picking point;

[0038] When the key point of the tender buds is a point, image segmentation is performed using a position and region growing algorithm based on P1 or P2 to obtain a binary image of the tea buds and branches. The noise therein is filtered out through expansion and corrosion manipulation, and the binary image is fit using a straight line fitting method. The key point is offset downward on the straight line by a preset distance before the position of the tea bud picking point is determined.

[0039] The present invention provides a method for locating tea bud picking points based on a deep learning algorithm. Through the above method, the present invention can determine the picking points according to the growth posture of the tea buds, ensure that the located picking point coordinates all fall on the petioles of the tea buds, improve the integrity of the picked tea buds, reduce the influence of the surrounding environment on its positioning, and improve the accuracy and efficiency of tea bud picking point positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of the first embodiment of the method for locating tea bud picking points based on a deep learning algorithm of the present invention;

[0041] Figure 2 The present invention is a flow chart of the method for locating tea bud picking points based on a deep learning algorithm.

[0042] Figure 3 It is a schematic diagram of the key point positions of the tea bud picking point positioning method based on the deep learning algorithm of the present invention;

[0043] Figure 4 A thermal image of tea buds taken by the tea bud picking point positioning method based on a deep learning algorithm of the present invention;

[0044] Figure 5 The RGB image of tea buds taken by the tea bud picking point positioning method based on the deep learning algorithm of the present invention;

[0045] Figure 6 It is a labeled image of an image in a tea bud semantic segmentation database of a tea bud picking point positioning method based on a deep learning algorithm of the present invention;

[0046] Figure 7 This is the effect diagram after the present invention performs the semantic segmentation model on the tea bud thermal image and the tea bud RGB image;

[0047] Figure 8This is the minimum bounding rectangle effect diagram after executing the semantic segmentation model on the tea bud thermal image and the tea bud RGB image;

[0048] Fig. 9 The minimum bounding rectangle effect diagram displayed on the tea buds RGB image after executing the semantic segmentation model on the tea buds thermal image and the tea buds RGB image;

[0049] Fig.10 This is a flow chart of step S71 of the method for locating tea bud picking points based on a deep learning algorithm of the present invention;

[0050] Fig.11 This is a flow chart of step S72 of the method for locating tea bud picking points based on a deep learning algorithm of the present invention.

[0051] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] Reference Figure 1 , Figure 1 This is a flow chart of the first embodiment of the method for locating tea bud picking points based on a deep learning algorithm of the present invention.

[0053] In an embodiment of the present invention, the tea bud picking point positioning method based on the deep learning algorithm is applied to a bud picking point positioning device, and the method includes:

[0054] Step S10, obtaining a pair of tea bud images of a plurality of tea tree buds, wherein the pair of tea bud images includes a tea bud thermal image and a tea bud RGB image;

[0055] In this embodiment, in order to improve the accuracy and efficiency of tea bud picking point positioning, the bud picking point positioning device uses an image acquisition device to capture several pairs of tea bud images from the side of the buds at a height close to the tea buds, perpendicular to the buds, to achieve tea bud semantic segmentation data collection and tea bud key point detection data collection; the tea bud image pair includes a tea bud thermal image and a tea bud RGB image, the tea bud thermal image and the tea bud RGB image are side views of the tea buds, wherein the tea bud RGB image is a color picture of the tea buds, which is an image composed of the three primary colors of red, green and blue.

[0056] Step S20, performing semantic segmentation, key point detection and labeling on each pair of tea bud images, to obtain a tea bud semantic segmentation database and a tea bud key point detection database;

[0057] In this embodiment, after acquiring a plurality of tea bud image pairs of tea tree buds, the bud picking point positioning device labels each pair of the tea bud image pairs to obtain a tea bud semantic segmentation database and a tea bud key point detection database.

[0058] Step S20 labels each pair of tea bud images according to the formats of input data of two types of tasks, namely semantic segmentation and key point detection, to obtain a tea bud semantic segmentation database and a tea bud key point detection database, which may include:

[0059] Step S21, annotating each pair of the tea bud images to obtain semantic segmentation annotation data and key point annotation data corresponding to the tea bud image pair;

[0060] In this embodiment, after obtaining a pair of tea bud images of several tea tree buds, the bud picking point positioning device respectively annotates them according to the formats of two types of task input data: semantic segmentation and key point detection, and obtains semantic segmentation annotation data and key point annotation data corresponding to the tea bud image pairs. Among them, when annotating semantic segmentation, the area where the tea buds are located on the image is annotated, and each tea bud is annotated as 1, and the rest of the area is annotated as 0; Figure 6 This is a labeled diagram of images in the tea bud semantic segmentation database of the tea bud picking point positioning method based on the deep learning algorithm of the present invention. For the convenience of visualization, the diagram is only displayed on the tea bud RGB image. The key point detection is labeled to label the collected data according to the key point detection task; for key point detection, two key points on the tea buds are labeled, the bud-leaf connection point P1 of the tea bud with one bud and one leaf, and the connection point P2 of the bud and leaf corresponding to one bud and two leaves and the branch. The labeling result of each group of buds is the information of the two key points P1 and P2 {(Px1, Py1, V1), (Px2, Py2, V)}, where the first two numbers are the position coordinate information, and the third number represents the visibility flag, 0 means unlabeled, 1 means labeled but invisible, and 2 means labeled and visible.

[0061] Step S22, performing augmentation operations on the semantic segmentation annotation data and the key point annotation data respectively to obtain a tea bud semantic segmentation database and a tea bud key point detection database.

[0062] In this embodiment, after obtaining the semantic segmentation annotation data and the key point annotation data, the bud picking point positioning device performs an augmentation operation on the semantic segmentation annotation data and the key point annotation data to achieve data set sample expansion and construct a tea bud semantic segmentation database Ds and a tea bud key point detection database Dk. The augmentation operation includes operations such as image translation, rotation, sharpening, flipping, and scaling.

[0063] Step S30, inputting the sprout semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a semantic segmentation model;

[0064] In this embodiment, after obtaining the tea bud semantic segmentation database and the tea bud key point detection database, the bud picking point positioning device inputs the bud semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a trained semantic segmentation model.

[0065] Step S30 inputs the sprout semantic segmentation database into a deep learning semantic segmentation model for training to obtain a semantic segmentation model weight, which may include:

[0066] Step S31, dividing the sprout semantic segmentation database into a semantic segmentation training set and a semantic segmentation verification set according to a first preset ratio;

[0067] In this embodiment, after obtaining the tea bud semantic segmentation database and the tea bud key point detection database, the bud picking point positioning device divides the bud semantic segmentation database into a semantic segmentation training set and a semantic segmentation verification set according to a first preset ratio; wherein the first preset ratio is 8:2.

[0068] Step S32, inputting the semantic segmentation training set into a deep learning semantic segmentation model for training to obtain a semantic segmentation model weight file;

[0069] In this embodiment, after obtaining the semantic segmentation training set, the bud picking point positioning device inputs the semantic segmentation training set into the deep learning semantic segmentation model for training to obtain the semantic segmentation model weight file; and verifies the trained deep convolutional neural network model through the semantic segmentation verification set. The deep learning semantic segmentation model is the FCN2s model.

[0070] Step S33, loading the trained target detection model weight file into the target detection model to obtain the trained target detection model.

[0071] In this embodiment, after obtaining the semantic segmentation model weight file, the bud picking point positioning device loads the trained semantic segmentation model weight file into the semantic segmentation model to obtain the trained semantic segmentation model. The semantic segmentation model FCN2s and the corresponding semantic segmentation model weight file together constitute the trained semantic segmentation model Fs, which can obtain the coordinates S of n tea bud regions on the image.

[0072]

[0073] Step S40, inputting the sprout key point detection database into the key point detection model for training to obtain a trained key point detection model;

[0074] In this embodiment, after obtaining the tea bud semantic segmentation database and the tea bud key point detection database, the bud picking point positioning device inputs the bud key point detection database into the key point detection model for training to obtain the trained key point detection model.

[0075] Step S40 inputs the sprout key point detection database into the key point detection model to obtain the key point detection model, which may include:

[0076] Step S41, dividing the sprout key point detection database into a key point training set and a key point verification set according to a second preset ratio;

[0077] In this embodiment, after obtaining the tea bud semantic segmentation database and the tea bud key point detection database, the bud picking point positioning device divides the bud key point detection database into a key point training set and a key point verification set according to a second preset ratio; wherein the second preset ratio is 8:2.

[0078] Step S42, inputting the key point training set into the key point detection model for training to obtain the key point detection model;

[0079] In this embodiment, after obtaining the key point training set, the tender shoot picking point positioning device inputs the key point training set into the key point detection model for training to obtain the key point detection model weight file; and verifies the key point detection model through the key point verification set.

[0080] Step S42 inputs the key point training set into the key point detection model for training to obtain the tea key point detection model weight file and the key point detection model, which may include:

[0081] Step S421, using the pytorch program to establish a key point detection model based on the HRNet network, wherein the improved key point detection model is to improve the input layer of the key point detection model to 4 channels for inputting the image after the thermal image and the RGB image are fused;

[0082] In this embodiment, after obtaining the key point training set, the tender bud picking point positioning device establishes a key point detection model based on the HRNet network using the pytorch program. The improved key point detection model improves the input layer of the key point detection model to 4 channels for inputting images fused from thermal images and RGB images. Among them, the key point detection model is based on the HRNet network, and the model outputs 2 feature maps, which respectively represent the prediction maps of the key points P1 and P2, and the loss function of the key point detection model redesigned using the key point features of tea tender buds is: min L = λ1*L1+λ2*L2+λ3*L3;

[0083]

[0084] L2=d(P1,C);

[0085] L3=d(P2,C);

[0086] Among them, λ1, λ2, λ3 are the weight coefficients of L1, L2, L3; Pk represents the kth key point of the sample, Represents the key point heat map predicted by the network, y(P k ) represents the heat map obtained by the true value; d(P1,C) represents the Euclidean distance from the key point of the tea bud predicted by the network to the center C of the rectangular area where it is located. The loss function of the key point detection model is used for training the key point detection model.

[0087] Step S422, inputting the key point training set into the key point detection model in batches and continuously and iteratively, when the model converges, the training of the key point detection model is completed, and a key point detection model weight file is obtained;

[0088] In this embodiment, after the key point detection model is established, the bud picking point positioning device inputs the key point training set into the key point detection model in batches and continuously and iteratively. As the iteration proceeds, the model will gradually converge, and finally complete the training of the key point detection model to obtain the key point detection model weight file. During the training process, when the loss function of the training set tends to be stable and the decline is small, the model is considered to have converged, that is, the training is completed. In this model, when the difference between the loss functions of three consecutive epochs is less than 0.1 during the training process, the training is stopped.

[0089] An iterative process in model training includes forward propagation and back propagation. In forward propagation, a batch of image data samples in the training set are input into the key point detection model, and then continuously propagated backward through the convolutional layer in the model, and the output result of the model is finally obtained. The loss function of the model is calculated by comparing the output result of the model with the label file; in back propagation, the above loss function is used to calculate the partial derivatives of the parameters in the convolutional layer, and the gradient is back propagated to update the weight parameters of each convolutional layer in the model, thereby realizing model training.

[0090] The key point detection model and the corresponding weight file together constitute the function Fk, which can obtain the positions of key points P1 and P2;

[0091]

[0092] Step S43, loading the trained key point detection model weight file into the key point detection model to obtain the trained key point detection model.

[0093] In this embodiment, after obtaining the key point detection model weight file, the tender shoot picking point positioning device loads the trained key point detection model weight file into the key point detection model to obtain the trained key point detection model; and verifies the key point detection model through the key point verification set.

[0094] Step S50, using an image acquisition device to capture an image pair of young tea buds to be picked;

[0095] In this embodiment, after obtaining the trained semantic segmentation model and key point detection model, the bud picking point positioning device uses an image acquisition device to shoot an image pair I of the tea buds to be picked. The image pair is a thermal image and a color RGB image of the tea buds to be picked.

[0096] Step S60, obtaining the key point positions of the sprouts according to the trained semantic segmentation model, the key point detection model and the image pairs;

[0097] In this embodiment, after obtaining the image pair I, the tender shoot picking point positioning device obtains the tender shoot key point position based on the semantic segmentation model, the key point detection model and the image pair.

[0098] Step S60 obtains the key point positions of the sprouts according to the trained semantic segmentation model, the key point detection model and the image pair, which may include:

[0099] Step S61, performing semantic segmentation on the image pair using the trained semantic segmentation model to obtain a tea bud segmentation result;

[0100] In this embodiment, after obtaining the image pair I, the bud picking point positioning device performs semantic segmentation on the image pair through the trained semantic segmentation model to obtain the tea bud segmentation result. Figure 7 As shown, the semantic segmentation model and the corresponding semantic segmentation model weight file are used to form a trained semantic segmentation model Fs to perform semantic segmentation on the image pair, obtain the corresponding tea bud semantic segmentation area, calculate the minimum bounding rectangle of the bud area, crop the area where the minimum bounding rectangle area is located from the original image, adjust it to a fixed resolution (128*224), and obtain the tea bud detection result; wherein, Figure 7 This is the effect diagram after the present invention performs the semantic segmentation model on the thermal image of tea buds and the RGB image of tea buds. In the figure, white represents the bud area and black represents the background.

[0101] Step S62, performing key point detection on the tea bud segmentation result using the trained key point detection model to obtain key point positions.

[0102] In this embodiment, after obtaining the tea bud segmentation result, the bud picking point positioning device performs key point detection on the tea bud detection image pair through the trained key point detection model to obtain the key point position. Figure 8 and 9 As shown, the key point detection model Fk is constructed by using the key point detection model and the corresponding key point detection model weight file to detect the key points of the tea bud detection results obtained in the tea bud detection frame; Fig. 9 This is the minimum bounding rectangle effect diagram displayed on the tea bud RGB image after executing the semantic segmentation model on the tea bud thermal image and the tea bud RGB image. For the convenience of visualization, the figure is only displayed on the tea bud RGB image. The key point detection model and the corresponding key point detection weight file together constitute the function Fk, which can obtain the position of the key points P1 and P2;

[0103]

[0104] Step S70, positioning the tea tree bud picking point based on the bud growth characteristics and the bud key point positions to obtain the bud picking point positions.

[0105] In this embodiment, after obtaining the position of the key point of the tender bud, the tender bud picking point positioning device locates the tender buds of the tea tree according to the growth characteristics of the tender buds and the position of the key point of the tender buds to obtain the position of the tender bud picking point.

[0106] Step S70 locates the tea tree bud picking point according to the bud growth characteristics and the bud key point positions to obtain the bud picking point positions, which may include:

[0107] Step S71, when the sprout key points are two points, Fig.10 As shown, that is, P1 and P2 exist at the same time, and d is the distance between the two key points. P1 and P2 are used to establish a straight line equation. According to the characteristics that the tender bud picking point is always below the key point and close to the key point, combined with the key point position and the fitting straight line equation, the point on the line segment P1P2 with a distance of 0.4d from point P1 is taken as the tender bud picking point position of the tea leaves. Fig.10 This is a flow chart of step S71 of the method for locating tea bud picking points based on a deep learning algorithm of the present invention. For ease of visualization, the figure is only displayed on the RGB image of the tea buds. The square points in the figure represent the key points of the tea buds detected by the model, and the triangular points represent the tea bud picking points located by the present invention.

[0108] Step S72, when the sprout key point is a point, Fig.11 As shown, that is, there is only one P1 or P2, then the position and regional growing algorithm based on P1 or P2 is used for image segmentation to obtain a binary image of tea buds and branches. The noise is filtered out by dilation and corrosion manipulation, and the binary image is fitted by a straight line fitting method. Finally, according to the characteristics that the bud picking point is always below the key point and close to the key point, combined with the key point position and the fitted straight line equation, the key point is offset downward on the straight line for a certain distance to determine the position of the tea bud picking point, where the offset value is 5 pixels. Fig.11 This is a flow chart of step S72 of the method for locating tea bud picking points based on a deep learning algorithm of the present invention. For ease of visualization, the figure is only displayed on the tea bud RGB image. The top and bottom square points in the figure represent the key picking points of the tea buds detected by the present invention; the triangular points represent the tea bud picking points located by the present invention.

[0109] Step S73, when the number of key points of the young sprout obtained is zero, that is, no key points are detected, no subsequent operation is performed at this time, and the next set of image pairs is calculated.

[0110] Through the above scheme, this embodiment determines the picking points according to the growth posture of the tea buds, ensuring that the coordinates of the located picking points fall on the petioles of the tea buds, thereby improving the integrity of the picked tea buds, while reducing the impact of the surrounding environment on its positioning, and improving the accuracy and efficiency of the tea bud picking point positioning.

[0111] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0112] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0113] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0114] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for locating tea bud picking points based on a deep learning algorithm, characterized in that: The method comprises the following steps: Acquire a plurality of tea bud image pairs of tea tree buds, wherein the tea bud image pairs include a tea bud thermal image and a tea bud RGB image; Labeling each pair of tea bud images to obtain a tea bud semantic segmentation database and a tea bud key point detection database; Inputting the sprout semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a trained semantic segmentation model; Inputting the sprout key point detection database into the key point detection model for training to obtain a trained key point detection model; Using an image acquisition device to capture an image pair of young tea buds to be picked; According to the trained semantic segmentation model, key point detection model and image pairs, the key point positions of the sprouts are obtained; Positioning the tea tree bud picking point based on the bud growth characteristics and the bud key point positions to obtain the bud picking point positions; The step of locating the picking point of the tea tree buds in combination with the growth characteristics of the buds and the key point positions of the buds to obtain the position of the bud picking point comprises: When there are two key points of the tender bud, d is the Euclidean distance between the two key points. P1 and P2 are used to establish a straight line equation, and a point 0.4d away from point P1 is taken on the line segment P1P2 as the position of the tea bud picking point; When the key point of the tender buds is a point, image segmentation is performed using a position and region growing algorithm based on P1 or P2 to obtain a binary image of the tea buds and branches. The noise therein is filtered out through expansion and corrosion manipulation, and the binary image is fit using a straight line fitting method. The key point is offset downward on the straight line by a preset distance before the position of the tea bud picking point is determined.

2. The method for locating tea bud picking points based on a deep learning algorithm according to claim 1, characterized in that: The step of labeling each pair of tea bud images to obtain a tea bud semantic segmentation database and a tea bud key point detection database comprises: Each pair of tea bud images is labeled according to the formats of two types of task input data, namely semantic segmentation and key point detection, to obtain semantic segmentation labeling data and key point labeling data corresponding to the tea bud image pair; The semantic segmentation annotation data and the key point annotation data are augmented respectively to obtain a tea bud semantic segmentation database and a tea bud key point detection database.

3. The method for locating tea bud picking points based on a deep learning algorithm according to claim 1, characterized in that: The step of inputting the sprout semantic segmentation database into a semantic segmentation model based on deep learning for training to obtain a semantic segmentation model comprises: Dividing the sprout semantic segmentation database into a semantic segmentation training set and a semantic segmentation verification set according to a first preset ratio; Inputting the semantic segmentation training set into a deep learning semantic segmentation model for training to obtain a semantic segmentation model weight file; Load the trained semantic segmentation model weight file into the semantic segmentation model to obtain the trained semantic segmentation model.

4. The method for locating tea bud picking points based on a deep learning algorithm according to claim 1, characterized in that: The step of inputting the sprout key point detection database into the key point detection model for training to obtain the key point detection model comprises: Dividing the sprout key point detection database into a key point training set and a key point verification set according to a second preset ratio; Inputting the key point training set into the key point detection model for training to obtain a key point detection model weight file; The trained key point detection model weight file is loaded into the key point detection model to obtain the trained key point detection model.

5. The method for locating tea bud picking points based on a deep learning algorithm according to claim 4, characterized in that: The step of inputting the key point training set into the key point detection model for training to obtain the tea key point detection model weight file and the key point detection model includes: An improved key point detection model is established based on the HRNet network using the pytorch program. The improved key point detection model is to improve the input layer of the key point detection model to 4 channels for inputting the image after the thermal image and the RGB image are fused; The key point training set is input into the key point detection model in batches and continuously and iteratively. When the model converges, the training of the key point detection model is completed, and the key point detection model weight file is obtained.

6. The method for locating tea bud picking points based on a deep learning algorithm according to claim 5, characterized in that: The loss function of the key point detection model is: min L=λ1*L1+λ2*L2+λ3*L3; L2=d(P1,C); L3=d(P2,C); Among them, λ1, λ2, λ3 are the weight coefficients of L1, L2, L3; Pk represents the kth key point of the sample, represents the key point heat map predicted by the network, y(P k ) represents the heat map obtained from the true value; d(P1,C) represents the Euclidean distance from the key point of the tea bud predicted by the network to the center C of the rectangular area where it is located.

7. The method for locating tea bud picking points based on a deep learning algorithm according to claim 1, characterized in that: The step of obtaining the key point positions of the sprouts according to the trained semantic segmentation model, the key point detection model and the image pair comprises: The image pairs are semantically segmented using the trained semantic segmentation model to obtain the segmentation results of tea buds; The trained key point detection model is used to perform key point detection on the tea bud segmentation results to obtain the key point positions.

Images