U-net-based Semantic Segmentation Method and System for Tea Bud Recognition and Picking Point Location

Abstract

The present invention discloses a method and system for tea bud recognition and picking point positioning based on U-net semantic segmentation. By constructing an improved U-net semantic segmentation network structure and using a labeled tea bud dataset and an unlabeled tea bud dataset to train the improved U-net semantic segmentation network, semi-supervised learning is achieved. Tea bud pictures are taken in real time, and after preprocessing, they are input into the trained network model for feature extraction and enhanced feature extraction to obtain the semantic segmentation prediction result. The improved prediction head network involves an MSA module to achieve multi-scale channel attention. Finally, according to the semantic segmentation result, the coordinates of the tea bud picking point are located, realizing the real-time recognition of tea buds and the positioning of the picking point based on the improved U-net semantic segmentation model.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent tea picking, and particularly relates to an improved U-net semantic segmentation model construction method, as well as a tea tender bud recognition and picking point positioning method and a picking system based on the improved U-net semantic segmentation model. Background Art

[0002] Due to the popularization of tea picking mechanization, the tea picking efficiency has been significantly improved. However, current mechanical tea picking equipment can only roughly pick bulk tea and cannot selectively pick famous and high-quality tea. The picking of current famous and high-quality tea still relies on a large amount of labor. With the outflow of the young population and the prominent problem of population aging, the intelligent picking of famous and high-quality tea has become a research hotspot, and the research on the recognition of tender buds of famous and high-quality tea and the positioning of picking points has thus developed rapidly. On the one hand, based on traditional algorithms, the color of tea leaves is distinguished to identify tea tender buds, and the skeleton of the tender buds is extracted to locate the picking points. On the other hand, based on deep learning algorithms, the functions of tea tender bud recognition and picking point positioning are realized.

[0003] However, the above technologies still have the following defects: 1. The recognition and positioning of tea tender buds based on traditional algorithms are greatly affected by environmental factors, have low generalization ability, and high parameter tuning difficulty. It is not suitable for picking tea in real environments. 2. The recognition and positioning of tea tender buds based on deep learning, although maturely developed, require a large amount of labeled data sets, have high annotation costs, and low tea picking efficiency. Summary of the Invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a method and a system for tea tender bud recognition and picking point positioning based on U-net semantic segmentation.

[0005] The technical solution adopted by the present invention is as follows:

[0006] An improved U-net semantic segmentation model construction method, comprising the following steps:

[0007] Step 1: Collect multiple images of objects to be segmented, perform semantic segmentation annotation on a part of the images to obtain a labeled data set; the remaining part of the images is not subjected to semantic segmentation annotation to form an unlabeled data set;

[0008] Step 2: Construct an improved U-net semantic segmentation network structure, the improved U-net semantic segmentation network structure includes an encoder, a decoder, a semi-supervised training branch, and an improved prediction head;

[0009] The input of the encoder is the image to be semantically segmented, and the output of the encoder is respectively used as the input of the decoder and the input of the semi-supervised training branch;

[0010] The output of the decoder is used as the input of the improved prediction head;

[0011] The improved prediction head includes implementing a channel attention mechanism within the MSA module to output the semantic segmentation result;

[0012] The semi-supervised training branch includes a noise adder and K auxiliary decoders. The noise adder is used to add noise to the output of the encoder, and the result after adding noise is respectively input into the K auxiliary decoders; the semantic segmentation results output by the K auxiliary decoders are

[0013] Step 3: Use the labeled dataset and unlabeled dataset in Step 1 to train the constructed improved U-net semantic segmentation network.

[0014] Furthermore, the training method in Step 3 is as follows:

[0015] Step 3.1, use the labeled dataset According to each batch size, perform feature extraction through the encoder, and then perform enhanced feature extraction and upsampling through the decoder. For the semantic segmentation result output by the MSA and the label value y i calculate the cross-entropy loss to obtain the supervised loss L S , which is expressed as:

[0016]

[0017] Perform backpropagation on the improved U-net network and update the weight parameters in the encoder and decoder in a gradient descent manner;

[0018] Step 3.2, use the unlabeled data According to each batch size, perform feature extraction through the encoder, and parallelly input the result after feature extraction into the decoder and the noise adder; the semantic segmentation result output by the decoder is the output result of the noise adder is At the same time, is parallelly input into the K auxiliary decoders, and the semantic segmentation results output by the auxiliary decoders are Perform consistency regularization processing on the output results of the MSA and the auxiliary decoders, and calculate the unsupervised loss, which is expressed as:

[0019]

[0020] Perform backpropagation on the improved U-net network and update the weight parameters of the encoder, decoder, and auxiliary decoders in a gradient descent manner;

[0021] Step 3.3, calculate the semi-supervised loss after each epoch of training, expressed as:

[0022] L = L s + w u L u

[0023] where w u is an unsupervised loss weighting function.

[0024] A method for tea tender shoot recognition and picking point localization based on an improved U-net semantic segmentation algorithm, comprising the following steps:

[0025] Step 1, collect images of tea tender shoots to be recognized, and preprocess the collected tea tender shoot images; the preprocessing includes noise reduction, filtering, etc. of the tea tender shoot images;

[0026] Step 2, input the preprocessed tea tender shoot images into the improved U-net semantic segmentation model; output the semantic segmentation result; based on the above-trained improved U-net semantic segmentation model, perform semantic segmentation recognition on the input tea tender shoot images;

[0027] Step 3, perform edge scanning based on the semantic segmentation result to obtain the tender shoot edge, and calculate the tender shoot picking point based on the tender shoot edge;

[0028] Step 4, fuse the tender shoot edge and the tender shoot picking point.

[0029] Furthermore, the process of performing semantic segmentation recognition on the input tea tender shoot images is as follows:

[0030] Perform pruning on the semi-supervised training branch in the trained improved U-net semantic segmentation model; after pruning, the improved U-net semantic segmentation model only retains the encoder, decoder, and improved prediction head;

[0031] Take the tea tender shoot image as the input, and use the pruned improved U-net semantic segmentation model to perform semantic segmentation recognition, and output the semantic segmentation result.

[0032] Furthermore, scan the tea tender shoot contour in the semantic segmentation result to obtain all pixel coordinate points of the tea tender shoot contour, where the horizontal axis is the x-axis, the positive direction is from left to right, the vertical axis is the y-axis, and the positive direction is from top to bottom. Sort the pixel coordinate points of each tender shoot contour in the image according to the y coordinate value from large to small. The sorted pixel coordinate point set is D1 = {(x1, y1), …, (x m , y m )}; take the first n pixel point coordinate sets in the set as D2 = {(x1, y1), …(x n , y n )};

[0033] Among them, n is an integer and n < m. The value of n is determined by the corresponding young sprout contour area, and the specific expression is as follows:

[0034]

[0035] Among them, M["m00"] is the area enclosed by each young sprout contour, and c is a fixed parameter value, which can be determined according to the specific tea variety and different internal camera parameters. Since M["m00"] / c is not an integer in most cases, its result is rounded down, and in the formula is the floor symbol;

[0036] Calculate the pixel coordinate average value for all points in the pixel coordinate set D2 = {(x1, y1), … (x n , y n )}, and the specific expression is as follows

[0037]

[0038] The calculated x and y are the pixel coordinate points of the picking points of high-quality tea young sprouts. Combining the internal parameters of the depth camera, the actual spatial coordinates (x, y, z) of the young sprout picking points can be calculated based on this coordinate point (x, y).

[0039] Furthermore, edge scanning adopts the method of calling the cv2.findContours sub-function in OpenCV.

[0040] Furthermore, call the Image.blend sub-function in the PIL library to realize the fusion of the young sprout edge and the young sprout picking point.

[0041] A tea young sprout recognition and picking point positioning system based on an improved U-net semantic segmentation algorithm; the system includes an image acquisition module, an image preprocessing module, a semantic segmentation module, a picking point calculation module, an execution unit and a control unit;

[0042] The image acquisition module is used to acquire the image of the tea young sprout to be recognized; the image acquisition module is signal-connected to the image preprocessing module and inputs the acquired image into the image preprocessing module;

[0043] The image preprocessing module is used to perform preprocessing such as noise reduction and filtering on the acquired image; the image preprocessing module is signal-connected to the semantic segmentation module and inputs the preprocessed image into the semantic segmentation module;

[0044] The semantic segmentation module incorporates a pre-trained improved U-net semantic segmentation model, which is used to perform semantic segmentation of tea buds on the pre-processed image; the semantic segmentation module is signal-connected to the picking point positioning module, and the result of the semantic segmentation is input into the picking point calculation module;

[0045] The picking point calculation module incorporates an edge scanning algorithm and a picking point algorithm. The picking point calculation module calculates the picking points of tea buds based on the semantic segmentation result; the picking point calculation module is signal-connected to the controller, and the edge scanning result and the calculated picking points of tea buds are input into the controller;

[0046] The controller is signal-connected to the execution unit, which is a manipulator, and performs the picking action of tea leaves under the control of the controller.

[0047] Furthermore, the image acquisition module selects a depth camera.

[0048] Furthermore, the image pre-processing module incorporates the K-SVD and morphological filtering algorithms.

[0049] Advantages of the present invention:

[0050] (1) In this application, a semi-supervised training branch is added to the U-net network. In addition to the encoder and decoder in the semi-supervised training branch, an auxiliary decoder is also added, which enables the network to use a large amount of unlabeled data to train the network model, realizing semi-supervised learning, greatly reducing the manual annotation cost, and effectively preventing overfitting of the network model.

[0051] (2) Adding the MSA module to the U-net prediction head enables the network to achieve multi-scale channel attention, making the network prediction unaffected by the size of the acquired image, and improving the recognition accuracy of the network for small objects such as tea buds. Compared with the traditional attention mechanism, the number of parameters is less and the network is more lightweight.

[0052] (3) Applying the improved U-net semantic segmentation network to the recognition and positioning of tea leaves simplifies the problem of tea leaf recognition and positioning, avoids problems such as high difficulty in adjusting parameters of traditional algorithms and low picking rates of existing deep learning algorithms, and improves the efficiency of tea bud recognition and picking point positioning. Description of the Drawings

[0053] Figure 1 is the overall technical flow chart of the present invention;

[0054] Figure 2 is the structural diagram of the improved U-net network described in the present invention;

[0055] Figure 3 is the flow chart of the semi-supervised learning training strategy described in the present invention;

[0056] Figure 4 This is the MSA module of the present invention;

[0057] Figure 5 This is the specific flowchart of the picking point positioning module of the present invention. Detailed implementation manners

[0058] In order to make the objectives, technical solutions and advantages of the present invention more clear and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] An improved method for constructing a U-net semantic segmentation model includes the following steps:

[0060] Step 1: Take multiple images of the object to be segmented from different angles and at different distances; in this embodiment, the tea tender buds are used as the object to be segmented, so all the collected images are of tea tender buds; perform semantic segmentation annotation on a part of the tea tender bud images to obtain a labeled data set; the remaining tea tender bud images are not subjected to semantic segmentation annotation to form an unlabeled data set. In this embodiment, the lamelbe software can be used to perform semantic segmentation annotation on the images.

[0061] Among them, the labeled data set is prepared for supervised learning, and the unlabeled data set is prepared for unsupervised learning. The combined learning method of supervised learning and unsupervised learning is the semi-supervised learning training strategy of the present invention; it can effectively prevent the model from overfitting, improve the model performance, and reduce the semantic segmentation annotation cost.

[0062] Step 2: Construct an improved U-net semantic segmentation network structure. As Figure 2 shown, the improved U-net network generally presents a U-shaped structure, which mainly includes an encoder, a decoder, a semi-supervised training branch, and an improved prediction head; specifically:

[0063] The encoder is used for feature extraction, and the tea tender bud image is used as the input of the encoder; the output of the encoder (i.e., the image features extracted by the encoder) is respectively used as the input of the decoder and the input of the semi-supervised training branch; in this embodiment, the input tea tender bud image can be preprocessed such as filtering and noise reduction.

[0064] The decoder is used to strengthen feature extraction, and the output of the decoder is used as the input of the improved prediction head; in this embodiment, the output of the decoder includes two parts, namely the semantic segmentation map after three upsamplings and the H*W*num_classes semantic segmentation map after four upsamplings.

[0065] The improved prediction head includes an MSA module, which takes the output of the decoder as the input of the MSA module, and the output of the MSA module serves as the semantic segmentation result of the final output; as Figure 4 shown, the MSA module is specifically as follows: First, perform global average pooling on the semantic segmentation map, input the result after global average pooling into the BN layer, and output the output result of the BN layer in parallel; multiplying all channels of one output result by α (where 0 < α < 1) is denoted as g1, and multiplying all channels of the other output result by 1 - α is denoted as g2.

[0066] g1 is multiplied layer by layer with the semantic segmentation result, and then through one upsampling is denoted as q1, implementing the channel attention mechanism. g2 and the semantic segmentation result of H*W*num_classes are multiplied layer by layer and denoted as q2, and q1 and q2 are added as the semantic segmentation result.

[0067] Implement the multi-scale channel attention mechanism, which promotes that the network prediction result is not affected by the image size, and can improve the recognition accuracy of the network for small objects such as tea buds. Compared with the traditional attention mechanism, it has fewer parameters, making the network more lightweight.

[0068] As Figure 3 , the semi-supervised training branch includes a noise adder and K auxiliary decoders; the noise adder is used to add noise to the output of the encoder, and the result after adding noise is respectively input into K auxiliary decoders; the semantic segmentation results output by the K auxiliary decoders are (where 1 ≤ k ≤ K);

[0069] Step 3: Use the labeled dataset and the unlabeled dataset in Step 1 to train the constructed improved U-net semantic segmentation network; the process is as follows:

[0070] Step 3.1, as Figure 3 shown, is the semi-supervised training strategy flowchart of the present invention. Using the labeled dataset According to the size of each batch_size (batch), perform feature extraction through the encoder, and then perform enhanced feature extraction and upsampling through the decoder. Calculate the cross-entropy loss of the semantic segmentation result output by the MSA with the label value y i to calculate the supervised loss L S , specifically as the following expression:

[0071]

[0072] Backpropagation is performed on the improved U-net network to update the weight parameters in the encoder and decoder in a gradient descent manner.

[0073] Step 3.2, Use the data without labels in the dataset (where m >> n), according to the size of each batch_size, perform feature extraction through the encoder, and input the results after feature extraction into the decoder and the noise adder in parallel. The semantic segmentation result output by the decoder is The output result of the noise adder is At the same time, Input in parallel into K auxiliary decoders, and the semantic segmentation result output by the auxiliary decoder is (where 1 ≤ k ≤ K). Perform consistency regularization processing on the output results of the MSA and the auxiliary decoder, and calculate the unsupervised loss, as shown in the following expression:

[0074]

[0075] Backpropagation is performed on the improved U-net network to update the weight parameters of the encoder, decoder, and auxiliary decoder in a gradient descent manner.

[0076] Step 3.3, Calculate the semi-supervised loss after each epoch of training, as shown in the following expression:

[0077] L = L s + w u L u

[0078] where, w u is an unsupervised loss weighting function, w u starts from 0, continuously increases, and becomes a fixed value w after a certain number of epochs, aiming to avoid obtaining unstable predictions at the beginning of training.

[0079] The semi-supervised learning of the present invention can optimize the network model by using unlabeled data, reduce the dependence of the network model on labeled data, reduce the manual annotation cost, and better adapt to the intelligent recognition of different tea gardens and different types of tea.

[0080] Combined with the attached Figure 1 and 5 shown a tea shoot recognition and picking point positioning method based on an improved U-net semantic segmentation algorithm, specifically including the following steps:

[0081] Step 1, Collect images of tea shoots to be recognized, and preprocess the collected tea shoot images; the preprocessing includes processing such as noise reduction and filtering of the tea shoot images;

[0082] Step 2: Input the preprocessed tea bud image into the improved U-net semantic segmentation model; output the semantic segmentation result; based on the above-trained improved U-net semantic segmentation model, perform semantic segmentation recognition on the input tea bud image.

[0083] The specific process is as follows:

[0084] First, perform pruning on the semi-supervised training branch in the trained improved U-net semantic segmentation model; after pruning, the improved U-net semantic segmentation model only retains the encoder, decoder, and improved prediction head;

[0085] Take the tea bud image as the input, and use the pruned improved U-net semantic segmentation model to perform semantic segmentation recognition and output the semantic segmentation result.

[0086] Step 3: Based on the semantic segmentation result, perform edge scanning to obtain the bud edge, and calculate the bud picking point based on the bud edge; the specific process is as follows:

[0087] Scan the tea bud contour in the semantic segmentation result to obtain all pixel coordinate points of the tea bud contour, where the horizontal axis is the x-axis, the positive direction is from left to right, the vertical axis is the y-axis, and the positive direction is from top to bottom. Sort the pixel coordinate points of each bud contour in the image according to the y coordinate value from large to small. The sorted pixel coordinate point set is D1 = {(x1, y1), …, (x m , y m )}. Take the first n pixel point coordinates in the set as D2 = {(x1, y1), …(x n , y n )}.

[0088] Among them, n is an integer and n < m. The value of n is determined by the corresponding bud contour area, specifically as follows:

[0089]

[0090] Among them, M["m00"] is the area enclosed by each bud contour, and c is a fixed parameter value, which can be determined according to specific tea varieties and different camera internal parameters. Since M["m00"] / c is not an integer in most cases, take the integer part of its result. In the formula is the floor function symbol.

[0091] Calculate the average value of the pixel coordinates for all points in the pixel coordinate set D2 = {(x1, y1), …(x n , y n ),) as follows:

[0092]

[0093] The calculated x and y are the pixel coordinate points of the picking points of the high-quality tea tender buds. Combining with the internal parameters of the depth camera, the actual spatial coordinates (x, y, z) of the tender bud picking points can be calculated according to these coordinate points (x, y).

[0094] In this embodiment, in step 3, edge scanning can adopt the method of calling the cv2.findContours sub-function by OpenCV, but it is not limited to this method, and other methods that can achieve the above-mentioned edge scanning can also be substituted.

[0095] Step 4: Integrate the edges of the tender buds and the picking points of the tender buds.

[0096] In this embodiment, in step 4, the Image.blend sub-function in the PIL library can be called to achieve the integration of the edges of the tender buds and the picking points of the tender buds.

[0097] In order to implement the above-mentioned tea tender bud recognition and picking point positioning method based on the improved U-net semantic segmentation algorithm, the present application also designs a tea tender bud recognition and picking point positioning system based on the improved U-net semantic segmentation algorithm; this system includes an image acquisition module, an image preprocessing module, a semantic segmentation module, a picking point calculation module, an execution unit, and a control unit.

[0098] The image acquisition module can select a depth camera; it is used to acquire images of the tea tender buds to be recognized; the image acquisition module is signal-connected to the image preprocessing module and inputs the acquired images into the image preprocessing module;

[0099] The image preprocessing module can internally integrate K-SVD and morphological filtering algorithms, and is used for preprocessing such as noise reduction and filtering of the acquired images; the image preprocessing module is signal-connected to the semantic segmentation module and inputs the preprocessed images into the semantic segmentation module;

[0100] The semantic segmentation module internally integrates a trained improved U-net semantic segmentation model, and uses the improved U-net semantic segmentation model to perform semantic segmentation of the tea tender buds on the preprocessed images; the semantic segmentation module is signal-connected to the picking point positioning module and inputs the results of the semantic segmentation into the picking point calculation module;

[0101] The picking point calculation module internally integrates an edge scanning algorithm and a picking point algorithm, and the picking point calculation module calculates the picking points of the tea tender buds based on the semantic segmentation results; the picking point calculation module is signal-connected to the controller and inputs the edge scanning results and the calculated picking points of the tea tender buds into the controller;

[0102] The controller is signal-connected to the execution unit which is a manipulator, and performs the picking action of the tea under the control of the controller.

[0103] The above embodiments are only used to illustrate the design concept and features of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design concepts disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A method for identifying tea tender buds and locating picking points based on an improved U-net semantic segmentation algorithm, characterized in that, It includes the following steps: Step 1: Collect images of tea buds to be recognized, and preprocess the collected tea bud images; the preprocessing includes noise reduction and filtering of the tea bud images; Step 2: Input the preprocessed tea bud images into the improved U-net semantic segmentation model; output the semantic segmentation results; the construction method of the improved U-net semantic segmentation model is as follows: Step 2.1: Collect multiple images of objects to be segmented, perform semantic segmentation annotation on a part of the images to obtain a labeled dataset; the remaining part of the images is not subjected to semantic segmentation annotation to form an unlabeled dataset; Step 2.2: Construct an improved U-net semantic segmentation network structure, and the improved U-net semantic segmentation network structure includes an encoder, a decoder, a semi-supervised training branch, and an improved prediction head; The input of the encoder is the image to be semantically segmented, and the output of the encoder is respectively used as the input of the decoder and the input of the semi-supervised training branch; The output of the decoder is used as the input of the improved prediction head; The improved prediction head includes implementing a channel attention mechanism within the MSA module and outputting the semantic segmentation results; The semi-supervised training branch includes a noise adder and K auxiliary decoders. The noise adder is used to add noise to the output of the encoder, and the result after adding noise is respectively input into the K auxiliary decoders; the semantic segmentation results output by the K auxiliary decoders are Step 2.3: Use the labeled dataset and the unlabeled dataset in Step 2.1 to train the constructed improved U-net semantic segmentation network; Step 3: Perform edge scanning based on the semantic segmentation results to obtain the bud edge, and calculate the bud picking point based on the bud edge, specifically including: Scan the outline of the tea buds in the semantic segmentation result to obtain all the pixel coordinate points of the tea bud outline. The horizontal axis is the x-axis, with the positive direction from left to right, and the vertical axis is the y-axis, with the positive direction from top to bottom. Sort the pixel coordinate points of each bud outline in the image in descending order according to the y coordinate value. The sorted set of pixel coordinate points is D1 = {(x1, y1),…,(x m ,y m )}; Take out the first n pixel point coordinate sets in the set as D2 = {(x1, y1),…(x n ,y n )}; where n is an integer and n < m, and the value of n is determined by the area of the corresponding bud outline, as shown in the following expression: Among them, M["m00"] is the area enclosed by each tender bud contour, c is a fixed value parameter, which is determined according to specific tea varieties and different internal camera parameters; since M["m00"] / c is not an integer in most cases, the result is rounded down. In the formula, is the floor function symbol; For all points in the pixel coordinate set D2 = {(x1, y1), …(x n , y n )}, calculate the average pixel coordinates, as shown in the following expression: The calculated x and y are the pixel coordinate points of the picking points of the famous tea buds. Combining the internal parameters of the depth camera, the actual spatial coordinates (x, y, z) of the bud picking point are calculated according to this coordinate point (x, y); Step 4: Fuse the bud edge and the bud picking point.

2. The tea shoot recognition and picking point positioning method based on the improved U-net semantic segmentation algorithm according to claim 1, characterized in that, The process of semantic segmentation and recognition of the input tea bud images is as follows: Perform pruning processing on the semi-supervised training branch in the trained improved U-net semantic segmentation model; After pruning, the improved U-net semantic segmentation model only retains the encoder, the decoder, and the improved prediction head; Use the tea bud image as the input, and use the pruned improved U-net semantic segmentation model to perform semantic segmentation and recognition, and output the semantic segmentation results.

3. The method for identifying tea tender buds and locating picking points based on an improved U-net semantic segmentation algorithm according to claim 1, wherein, The training method is as follows: Step 2.3.1, using the labeled dataset According to each batch size, perform feature extraction through the encoder, and then perform enhanced feature extraction and upsampling through the decoder on the semantic segmentation result output by the MSA and the label value y i Perform cross-entropy loss calculation to supervise the loss L S , expressed as: Perform backpropagation on the improved U-net network and update the weight parameters in the encoder and decoder in the form of gradient descent; Step 2.3.2, using the data without labels According to each batch size, feature extraction is performed through the encoder, and the results after feature extraction are input into the decoder and the noise adder in parallel; the semantic segmentation result output by the decoder is The output result of the noise adder is At the same time It is input into K auxiliary decoders in parallel, and the semantic segmentation result output by the auxiliary decoder is Perform consistency regularization processing on the output results of the MSA and the auxiliary decoder, and calculate the unsupervised loss, expressed as: Perform backpropagation on the improved U-net network and update the weight parameters of the encoder, decoder, and auxiliary decoder in the form of gradient descent; Step 2.3.3, calculate the semi-supervised loss after each epoch training, expressed as: L = L s + w u L u where, w u is an unsupervised loss weighting function.

4. The tea bud recognition and picking point positioning method based on the improved U-net semantic segmentation algorithm according to claim 1, characterized in that Edge scanning adopts the method of calling the cv2.findContours sub-function in OpenCV.

5. The tea bud recognition and picking point positioning method based on the improved U-net semantic segmentation algorithm according to claim 1, characterized in that, Call the Image.blend sub-function in the PIL library to implement the fusion of the bud edge and the bud picking point.

6. A tea tender shoot recognition and picking point positioning system based on an improved U-net semantic segmentation algorithm, characterized in that, Adopt the tea bud recognition and picking point positioning method based on the improved U-net semantic segmentation algorithm as described in Claim 1. The system includes an image acquisition module, an image preprocessing module, a semantic segmentation module, a picking point calculation module, an execution unit, and a control unit; The image acquisition module is used to acquire images of tea buds to be recognized; the image acquisition module is signal-connected to the image preprocessing module and inputs the acquired images into the image preprocessing module; The image preprocessing module is used to perform noise reduction and filtering preprocessing on the acquired images; the image preprocessing module is signal-connected to the semantic segmentation module and inputs the preprocessed images into the semantic segmentation module; The semantic segmentation module incorporates a trained improved U-net semantic segmentation model and uses the improved U-net semantic segmentation model to perform semantic segmentation of tea buds on the preprocessed images; The semantic segmentation module is signal-connected to the picking point positioning module and inputs the results of semantic segmentation into the picking point calculation module; The picking point calculation module incorporates an edge scanning algorithm and a picking point algorithm, and the picking point calculation module calculates the picking points of tea buds based on the results of semantic segmentation; The picking point calculation module is signal-connected to the controller and inputs the edge scanning results and the calculated picking points of tea buds into the controller; The controller is signal-connected to the execution unit, which is a manipulator, and the manipulator performs the tea picking action under the control of the controller.

7. A tea bud recognition and picking point positioning system based on an improved U-net semantic segmentation algorithm according to claim 6, characterized in that, The image acquisition module selects a depth camera.

8. The tea shoot recognition and picking point positioning system based on the improved U-net semantic segmentation algorithm according to claim 6, characterized in that, The image preprocessing module incorporates the K-SVD and morphological filtering algorithms.

Images