Automatic labeling method of tea buds based on deep learning

Abstract

The present application discloses a method for automatic labeling of tea buds based on deep learning, the method comprising collecting tea buds pictures and constructing a training data set; using the training data set to train an initial tea bud detection model to obtain a tea bud detection model; inputting tea buds pictures to be detected into the tea bud detection model, and outputting detection information through the tea bud detection model. The tea bud detection model in the embodiment of the present application automatically labels tea buds pictures, saving the labor cost and time cost of labeling tea buds pictures.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method for automatically labeling young tea buds based on deep learning. Background Art

[0002] The tea consumption market is becoming increasingly hot, and consumers' demand for high-quality tea is also increasing. At present, the picking of high-quality tea is mainly done manually, which is labor-intensive and inefficient. The use of robot mechanized picking can not only improve the picking efficiency, but also greatly reduce labor costs.

[0003] The visual module of the tea picking robot is one of its important components. Its recognition accuracy and speed determine the effect of subsequent robotic picking. When picking tea leaves, the tea picking robot needs to rely on tea buds pictures with annotation files. At present, tea buds pictures are generally annotated manually, which also requires a lot of manpower costs and time.

[0004] Therefore the prior art still needs to be improved and enhanced. Summary of the invention

[0005] The technical problem to be solved by this application is to provide a method for automatic labeling of tea buds based on deep learning in view of the shortcomings of the existing technology.

[0006] In order to solve the above technical problems, the first aspect of the present application provides a method for automatic labeling of tea buds based on deep learning, wherein the method for automatic labeling of tea buds based on deep learning specifically includes:

[0007] Collect tea buds pictures and build a training data set;

[0008] Using the training data set to train the initial tea bud detection model to obtain a trained tea bud detection model;

[0009] Using the tea bud detection model to annotate the tea bud picture to automatically generate an annotation file, wherein the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds;

[0010] Among them, the tea bud detection model adopts an improved yo lov5 network structure, and an SE attention module is added to the backbone network of the improved yo lov5 network structure. The neck network of the improved yo lov5 network structure includes two BiFPN structures. The front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yo lov5 network structure.

[0011] The method for automatic labeling of tea buds based on deep learning, wherein the SE attention module is located after the SPPE module in the backbone network, and the output item of the SE attention module is input into the SPPE module.

[0012] The method for automatic labeling of young tea buds based on deep learning, wherein the input items of the head network include the output items of the BiFPN structure located at the back and the image features extracted in the backbone network whose image scale is smaller than any image feature in the input items of the BiFPN structure located at the front.

[0013] The automatic labeling method for tea buds based on deep learning, wherein the collecting of tea buds pictures and constructing a training data set specifically includes:

[0014] Photograph tea leaves at different angles in different weather conditions and at different time periods to collect multiple images of young tea buds;

[0015] Preprocessing a plurality of tea bud images to obtain a plurality of training tea bud images;

[0016] A plurality of training tea bud images are labeled with bud labeling frames and bud categories corresponding to the bud labeling frames so that each training tea bud image carries labeling information, wherein the bud categories include one leaf and one bud and one leaf and two buds.

[0017] The method for automatic labeling of tea buds based on deep learning, wherein the initial tea bud detection model is trained using the training data set to obtain the trained tea bud detection model specifically comprises:

[0018] Preprocessing the training data set to obtain a preprocessed training data set, wherein the preprocessing includes one or more of Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling processing;

[0019] Inputting the training tea bud images in the preprocessed training data set into the initial tea bud detection model, and outputting prediction information through the initial tea bud detection model;

[0020] Based on the annotation information carried by the training tea buds picture and the prediction information, the initial tea buds detection model is trained to obtain a trained tea buds detection model.

[0021] The method for automatic labeling of tea buds based on deep learning, wherein the training tea buds images in the preprocessed training data set are input into the initial tea buds detection model, and outputting prediction information through the initial tea buds detection model specifically includes:

[0022] Inputting the training tea bud images in the preprocessed training data set into the backbone network, and inputting the multi-scale feature map through the backbone network;

[0023] Inputting the multi-scale feature map into the front BiFPN structure in the neck network, and fusing the multi-scale feature map through the front BiFPN structure to obtain a multi-scale fused feature map;

[0024] Inputting the multi-scale fusion feature map into the BiFPN structure located at the back in the neck network, and fusing the multi-scale fusion feature map through the BiFPN structure located at the back to obtain a multi-scale target feature map;

[0025] The multi-scale target feature map is input into a head network, and the prediction information is output through the head network.

[0026] The method for automatic labeling of tea buds based on deep learning, wherein the initial tea bud detection model is trained based on the labeling information carried by the training tea bud picture and the prediction information to obtain a trained tea bud detection model specifically includes:

[0027] Constructing a loss function term based on the annotation information carried by the training tea bud image and the prediction information;

[0028] Training the initial tea bud detection model based on the loss function term to obtain a trained tea bud detection model;

[0029] Wherein, the loss function term is expressed as:

[0030] CIoU_Loss=1-IoU+αν+β(d / c)∧2

[0031]

[0032] Where v represents the correction factor, α and β represent adjustment coefficients, v represents the correction factor, d represents the Euclidean distance between the center point of the tender shoot annotation box in the annotation information and the center point of the prediction box in the prediction information, c represents the diagonal distance of the minimum circumscribed rectangle that contains both the prediction box and the tender shoot annotation box, IoU represents the intersection over union ratio of the annotation box and the prediction box, and w represents the intersection over union ratio of the annotation box and the prediction box. gt Indicates the width of the sprout annotation box in the annotation information, h gt represents the height of the sprout annotation box in the annotation information, w represents the width of the prediction box in the prediction information, and h represents the height of the prediction box in the prediction information.

[0033] The second aspect of the present application provides a tea bud detection device based on deep learning, wherein the tea bud detection device based on deep learning specifically includes:

[0034] The collection module is used to collect tea buds pictures and build a training data set;

[0035] A training module, used to train the initial tea bud detection model using the training data set to obtain a trained tea bud detection model;

[0036] A control module is used to use the tea bud detection model to annotate the tea bud picture to automatically generate an annotation file, wherein the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds; the tea bud detection model adopts an improved yolov5 network structure, and an SE attention module is added to the backbone network of the improved yolov5 network structure, and the neck network of the improved yolov5 network structure includes two BiFPN structures, the front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yolov5 network structure.

[0037] Beneficial effects: Compared with the prior art, the present application provides a method for automatic labeling of tea buds based on deep learning, the method comprising collecting tea buds pictures and constructing a training data set; using the training data set to train an initial tea bud detection model to obtain a trained tea bud detection model; using the tea bud detection model to label the tea bud pictures to automatically generate a labeling file, wherein the labeling file comprises a labeling box and a labeling category, and the labeling category is one leaf and one bud or one leaf and two buds. The tea bud detection model in the embodiment of the present application automatically labels the tea bud pictures, saving the labor cost and time cost of labeling the tea bud pictures. At the same time, the tea bud detection model adopts the improved yo lov5 network structure, and the SE attention module is added to the backbone network of the improved yo lov5 network structure, and the neck network including the BiFPN structure is adopted as the neck network of the improved yo lov5 network structure, so that the importance weight of each channel in the feature map can be learned and determined by the SE attention module to improve the model performance of the tea bud detection model. At the same time, the neck network including the BiFPN structure can fuse features of different scales at different levels, realize forward and backward cross-layer feature transfer, further improve the model performance of the tea bud detection model, and thus further improve the accuracy of tea bud labeling. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0039] Figure 1 A flowchart of a method for automatic labeling of tea buds based on deep learning provided in an embodiment of the present application.

[0040] Figure 2 This is the principle flow chart of the backbone network.

[0041] Figure 3 Schematic diagram of the input items of the head network.

[0042] Figure 4 This is a principle block diagram of a tea bud detection device based on deep learning provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The embodiment of the present application provides a method for automatic labeling of tea buds based on deep learning. In order to make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0045] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0046] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0047] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0048] This embodiment provides a method for automatic labeling of tea buds based on deep learning. Figure 1 As shown, the method includes:

[0049] S10. Collect pictures of young tea buds and construct a training data set.

[0050] Specifically, the training data set includes a large number of tea bud pictures, for example, 2000, 3000, etc., each tea bud picture is used as a sample picture of the training data, and can be any form of tea bud pictures. Among them, the tea buds are divided into one leaf and one bud and one leaf and two buds, and each tea bud picture is marked with one leaf and one bud or one leaf and two buds, and one leaf and one bud and one leaf and two buds are used as bud categories, that is, each tea bud picture is marked with bud categories, and the bud categories include one leaf and one bud and one leaf and two buds.

[0051] Exemplarily, the collecting of tea bud images and constructing a training data set specifically includes:

[0052] Photograph tea leaves at different angles in different weather conditions and at different time periods to collect multiple images of young tea buds;

[0053] Preprocessing a plurality of tea bud images to obtain a plurality of training tea bud images;

[0054] A plurality of training tea bud images are labeled with bud labeling frames and bud categories corresponding to the bud labeling frames so that each training tea bud image carries labeling information, wherein the bud categories include one leaf and one bud and one leaf and two buds.

[0055] Specifically, different weather conditions may include sunny days, cloudy days, and rainy days, and different time periods may include morning, noon, evening, and night, wherein fill light is used for shooting at night. That is to say, the multiple pictures of tea buds include pictures of tea buds under different lighting conditions and different environmental interference factors. For example, the multiple pictures of tea buds include pictures of tea buds taken in the morning on a sunny day, pictures of tea buds taken in the evening on a cloudy day, and the like. In addition, in addition to shooting pictures of tea buds under different lighting conditions and different environmental interference factors, each time a picture of tea buds is shot, it will be shot in different shooting directions, and each shooting direction will also be shot at different shooting angles, wherein the shooting direction may include front shooting, overhead shooting, and side shooting, and the shooting angles may be 30°, 45°, and 90°, and the like.

[0056] After the plurality of tea bud images are captured, the plurality of tea bud images are preprocessed, wherein the preprocessing may include screening and cleaning, etc., and the tea bud images without obvious one-leaf bud or two-leaf bud features and the tea bud images that are not clearly distinguishable can be removed by the preprocessing. For example, the tea bud images whose buds are blocked by more than a preset threshold (e.g., 60%, 70%, etc.) can be removed by screening and cleaning.

[0057] After preprocessing, each of the several training tea bud pictures obtained by the preprocessing is labeled, so as to label each training tea bud picture with a bud labeling frame and a bud category corresponding to the bud labeling frame, wherein the bud labeling frame can be in the form of four-dimensional position coordinates, for example, the coordinate information of the upper left corner and the lower right corner of the bud labeling frame, or the center point of the bud labeling frame and the width and height of the labeling frame. In addition, after labeling several training tea bud pictures, the labeled tea bud pictures are divided into a training data set and a verification data set, for example, 2000 training tea bud pictures are labeled, and then the labeled 2000 training tea bud pictures are divided into a training data set and a verification data set according to 8:2, that is, 1600 training tea bud pictures are used as training data, and 400 training tea bud pictures are used as verification data. Among them, the training data is used for tea bud detection model training, and the verification data is used for verifying the trained tea bud detection model.

[0058] S20: using the training data set to train the initial tea bud detection model to obtain a trained tea bud detection model.

[0059] Specifically, the model structure of the initial tea bud detection model is the same as that of the tea bud detection model, and the network parameters of the two are different, wherein the network parameters of the initial tea bud detection model are the initialization network parameters, and the network parameters of the tea bud detection model are the network parameters trained by the training data set. That is to say, the initialization network parameters of the initial tea bud detection model are optimized by using the training data set to obtain the network parameters of the tea bud detection model.

[0060] In one implementation, the using the training data set to train the initial tea bud detection model to obtain the trained tea bud detection model specifically includes:

[0061] S21, preprocessing the training data set to obtain a preprocessed training data set, wherein the preprocessing includes one or more of Mosaic data enhancement, adaptive anchor frame calculation, and adaptive image scaling processing;

[0062] S22, inputting the training tea bud images in the preprocessed training data set into an initial tea bud detection model, and outputting prediction information through the initial tea bud detection model;

[0063] S23. Based on the annotation information carried by the training tea buds picture and the prediction information, the initial tea buds detection model is trained to obtain a trained tea buds detection model.

[0064] In step S21, Mosaic data enhancement is used to perform data enhancement on the training data set, wherein the process of Mosaic data enhancement can be to splice the 4 training tea buds pictures in the training data set by randomly scaling, randomly cropping, and randomly arranging them, and the specific process can be: randomly selecting the coordinates (x, y) of the splicing reference point of the training tea buds picture, and randomly selecting four other training tea buds pictures; the four training tea buds pictures are respectively resized and scaled according to the reference points, and placed in the upper left, upper right, lower left, and lower right positions of the large picture of the specified size; according to the size transformation method of each training tea bud picture, the mapping relationship is corresponded to the annotation information; according to the specified horizontal and vertical coordinates, the large picture is spliced, and the detection box coordinates that exceed the boundary are processed.

[0065] Adaptive anchor box calculation is used to calculate the prior box. Through adaptive anchor box calculation, each feature map contains multiple prior boxes of different scales. The size of the prior box will affect the convergence speed of the model and the accuracy of the prediction. To this end, in an embodiment of the present application, an adaptive anchor frame calculation is first performed to determine the prior frame. Specifically, the size of the bud annotation frame in all training tea bud images in the training data set is obtained; the bud annotation frame in each training tea bud image is scaled proportionally to obtain the scaled annotation frame size corresponding to each training tea bud image, so as to keep the maximum value of the width and height in the scaled annotation frame size less than or equal to the specified size; then the scaled annotation frame size is performed; the bboxes are changed from relative coordinates to absolute coordinates, where the scaled width and height are multiplied; the bboxes are screened to retain bboxes whose height is greater than or equal to a preset pixel value (for example, two pixels); k-means clustering is used to obtain n prior frames; a genetic algorithm is used to randomly mutate the width and height of the prior frame, and if the effect after the mutation is good, the mutated prior frame is used as the prior frame; if the effect after the mutation is worse, it is skipped and the mutation is performed a preset number of times (such as 1000, etc.), wherein the effect after the mutation can be evaluated based on the fitness calculated using the anchorfitness method.

[0066] Furthermore, the adaptive image scaling process is used to scale the image size of the training tea buds image to the input size of the initial tea buds detection model. The specific process can be to calculate the scaling ratio according to the image size of the training tea buds image and the input size of the initial tea buds detection model, and select a small scaling factor among different scaling factors; calculate the scaled image size according to the image size and the scaling ratio; calculate the black border filling value, and subtract the width and height in the scaled image size to obtain the height to be filled, and then use the pixel obtained by taking the modulus 32 to divide by 2 to obtain the value to be filled at both ends of the height in the image size, and fill it.

[0067] Further, in step S22, after preprocessing the training data set, the initial tea bud detection model is trained by the preprocessed training data set, wherein the tea bud detection model adopts an improved yolov5 network structure, the improved yo lov5 network structure includes a backbone network, a neck network and a head network, the neck network of the improved yo lov5 network structure includes two BiFPN structures, and the front BiFPN structure is connected to the backbone network.

[0068] Based on this, the step of inputting the training tea bud images in the preprocessed training data set into the initial tea bud detection model and outputting prediction information through the initial tea bud detection model specifically includes:

[0069] Inputting the training tea bud images in the preprocessed training data set into the backbone network, and inputting the multi-scale feature map through the backbone network;

[0070] Inputting the multi-scale feature map into the front BiFPN structure in the neck network, and fusing the multi-scale feature map through the front BiFPN structure to obtain a multi-scale fused feature map;

[0071] Inputting the multi-scale fusion feature map into the BiFPN structure located at the back in the neck network, and fusing the multi-scale fusion feature map through the BiFPN structure located at the back to obtain a multi-scale target feature map;

[0072] The multi-scale target feature map is input into a head network, and the prediction information is output through the head network.

[0073] Specifically, Figure 2 As shown, the backbone network includes a Focus structure, a CSP structure and an SPPE module. The CSP structure includes three convolutional layers and multiple Res un int modules, and a SE attention module is deployed between the last CSP and SPPE (spatial pyramid pooling module). The output item of the CSP located later is input into the SE attention module, and the output item of the SE attention module is input into the SPPE. This application adds an SE attention module between CSP and SPPE, and accurately locates and focuses on the key attention area through the SE attention module, thereby significantly improving the detection accuracy of the model. Among them, the core structure of the SE (Squeeze-and-Excitat ion) attention mechanism includes a global average pooling layer and two fully connected layers, which learn and determine the importance weight of each channel in the feature map through the training process to optimize the model performance. In addition, the SE mechanism does not introduce additional computational burden while improving performance. Specifically, the SE attention module contains only two basic operations, which makes it easy to embed into various convolutional neural network structures. By accurately capturing the importance information of each channel, the SE attention module can further enhance the performance of the network.

[0074] Furthermore, the neck network of the improved yo lov5 network structure includes two BiFPN structures, the front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yo lov5 network structure. For the sake of convenience, the BiFPN structure located in the front is recorded as the first BiFPN structure, and the BiFPN structure located in the back is recorded as the second BiFPN structure, wherein the input item of the BiFPN structure located in the front is the feature map extracted by some network layers in the backbone network. For example, the backbone network will extract five feature maps, which are recorded as P1, P2, P3, P4 and P5 respectively. The input items of the first BiFPN structure are P3, P4 and P5. After fusing P3, P4 and P5 through the first BiFPN structure, feature maps P31, feature map P41 and feature map P51 are obtained, and feature maps P31, feature map P41 and feature map P51 are input into the second Bi FPN structure, and feature maps P31, feature map P41 and feature map P51 are fused through the second BiFPN structure, and the fused feature maps P32, feature map P42 and feature map P52 are input into the head network. This application uses two BiFPN structures to establish bidirectional connections between different levels of the feature pyramid, realizes forward and backward cross-layer feature transfer, and enhances the model's semantic expression ability and ability to distinguish different features. At the same time, BiFPN also adopts a separate convolution and lightweight design strategy, which not only improves processing efficiency, but also effectively reduces the number of model parameters and computing requirements.

[0075] In addition to taking the output of the second BiFPN structure as input, the head network of the improved yo lov5 network structure also extracts image features whose image scale is smaller than any image feature in the input of the previous BiFPN structure. For example, in the above example, the backbone network also extracts feature maps P1 and P2, such as Figure 3 As shown, P2 can also be used as an input item of the head network, that is, the input item of the head network includes the output item of the second BiFPN structure and P2, so that the head network can obtain 4 different scale features (for example, feature maps of training tea buds images downsampled by 4 times, 8 times, 16 times and 32 times), thereby further improving the model accuracy of the tea bud detection model.

[0076] Further, in step S23, the prediction information includes a prediction box and a prediction category, the prediction box is the detection box of the tea buds predicted by the initial tea bud detection model, and the prediction category is the predicted bud category of the tea buds contained in the prediction box. To this end, after obtaining the prediction information, a loss function term can be constructed based on the prediction information and the labeling information, and then the network parameters can be optimized based on the loss function term.

[0077] Exemplarily, the training of the initial tea bud detection model based on the annotation information carried by the training tea bud picture and the prediction information to obtain the trained tea bud detection model specifically includes:

[0078] Constructing a loss function term based on the annotation information carried by the training tea bud image and the prediction information;

[0079] The initial tea bud detection model is trained based on the loss function term to obtain a trained tea bud detection model.

[0080] Specifically, the loss function term is expressed as:

[0081] CIoU_Loss=1-IoU+αν+β(d / c)∧2

[0082]

[0083] Where v represents the correction factor, α and β represent adjustment coefficients, v represents the correction factor, d represents the Euclidean distance between the center point of the tender shoot annotation box in the annotation information and the center point of the prediction box in the prediction information, c represents the diagonal distance of the minimum circumscribed rectangle that contains both the prediction box and the tender shoot annotation box, IoU represents the intersection over union ratio of the annotation box and the prediction box, and w represents the intersection over union ratio of the annotation box and the prediction box. gt Indicates the width of the sprout annotation box in the annotation information, h gt represents the height of the sprout annotation box in the annotation information, w represents the width of the prediction box in the prediction information, and h represents the height of the prediction box in the prediction information.

[0084] S30: using the tea bud detection model to annotate the tea bud image to automatically generate an annotation file.

[0085] Specifically, after obtaining the tea bud detection model, the tea bud detection model can be used to annotate the tea bud picture, so as to annotate the tea buds in the tea bud picture with an annotation box and an annotation category. That is, the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds. In addition, after the tea bud picture is annotated by the tea bud detection model, the annotated tea bud picture is transmitted to the tea picking robot, so that the robot can automatically pick the tea buds based on the tea bud picture carrying the annotation information.

[0086] In summary, the present embodiment provides a method for automatic annotation of tea buds based on deep learning, the method comprising collecting tea buds pictures and constructing a training data set; using the training data set to train the initial tea bud detection model to obtain a trained tea bud detection model; using the tea bud detection model to annotate the tea bud pictures to automatically generate an annotation file. The tea bud detection model in the embodiment of the present application automatically annotates the tea bud pictures, saving the labor cost and time cost of annotating the tea bud pictures. At the same time, the tea bud detection model adopts the improved yo lov5 network structure, and the SE attention module is added to the backbone network of the improved yo lov5 network structure, and the neck network including the BiFPN structure is used as the neck network of the improved yo lov5 network structure, so that the SE attention module can be used to learn and determine the importance weight of each channel in the feature map to improve the model performance of the tea bud detection model. At the same time, the neck network including the BiFPN structure can be used to fuse features of different scales at different levels, realize forward and backward cross-layer feature transfer, further improve the model performance of the tea bud detection model, and thus further improve the accuracy of tea bud annotation.

[0087] Based on the above-mentioned tea bud automatic labeling method based on deep learning, this embodiment provides a tea bud detection device based on deep learning, such as Figure 4 As shown, the tea bud detection device based on deep learning specifically includes:

[0088] A collection module 100 is used to collect tea bud pictures and construct a training data set;

[0089] A training module 200, for training an initial tea bud detection model using the training data set to obtain a trained tea bud detection model;

[0090] The control module 300 is used to use the tea bud detection model to annotate the tea bud picture to automatically generate an annotation file, wherein the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds; the tea bud detection model adopts an improved yo lov5 network structure, and an SE attention module is added to the backbone network of the improved yo lov5 network structure, and the neck network of the improved yo lov5 network structure includes two BiFPN structures, the front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yolov5 network structure.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatic labeling of tea buds based on deep learning, characterized in that: The automatic labeling method for tea buds based on deep learning specifically includes: Collect tea buds pictures and build a training data set; Using the training data set to train the initial tea bud detection model to obtain a trained tea bud detection model; Using the tea bud detection model to annotate the tea bud picture to automatically generate an annotation file, wherein the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds; Among them, the tea bud detection model adopts an improved yolov5 network structure, and an SE attention module is added to the backbone network of the improved yolov5 network structure. The neck network of the improved yolov5 network structure includes two BiFPN structures. The front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yolov5 network structure.

2. The method for automatic labeling of tea buds based on deep learning according to claim 1, characterized in that: The SE attention module is located after the SPPE module in the backbone network, and the output of the SE attention module is input into the SPPE module.

3. The method for automatic labeling of tea buds based on deep learning according to claim 1, characterized in that: The input items of the head network include the output items of the BiFPN structure located at the back and the image features extracted in the backbone network whose image scale is smaller than any image feature in the input items of the BiFPN structure located at the front.

4. The method for automatic labeling of tea buds based on deep learning according to claim 1, characterized in that: The collecting of tea bud images and constructing a training data set specifically includes: Photograph tea leaves at different angles in different weather conditions and at different time periods to collect multiple images of young tea buds; Preprocessing a plurality of tea bud images to obtain a plurality of training tea bud images; A plurality of training tea bud images are labeled with bud labeling frames and bud categories corresponding to the bud labeling frames so that each training tea bud image carries labeling information, wherein the bud categories include one leaf and one bud and one leaf and two buds.

5. The method for automatic labeling of tea buds based on deep learning according to claim 1, characterized in that: The step of training the initial tea bud detection model using the training data set to obtain the trained tea bud detection model specifically includes: Preprocessing the training data set to obtain a preprocessed training data set, wherein the preprocessing includes one or more of Mosaic data enhancement, adaptive anchor box calculation, and adaptive image scaling processing; Inputting the training tea bud images in the preprocessed training data set into the initial tea bud detection model, and outputting prediction information through the initial tea bud detection model; Based on the annotation information carried by the training tea buds picture and the prediction information, the initial tea buds detection model is trained to obtain a trained tea buds detection model.

6. The method for automatic labeling of tea buds based on deep learning according to claim 5, characterized in that: The step of inputting the training tea bud images in the preprocessed training data set into the initial tea bud detection model, and outputting prediction information through the initial tea bud detection model specifically includes: Inputting the training tea bud images in the preprocessed training data set into the backbone network, and inputting the multi-scale feature map through the backbone network; Inputting the multi-scale feature map into the front BiFPN structure in the neck network, and fusing the multi-scale feature map through the front BiFPN structure to obtain a multi-scale fused feature map; Inputting the multi-scale fusion feature map into the BiFPN structure located at the back in the neck network, and fusing the multi-scale fusion feature map through the BiFPN structure located at the back to obtain a multi-scale target feature map; The multi-scale target feature map is input into a head network, and the prediction information is output through the head network.

7. The method for automatic labeling of tea buds based on deep learning according to claim 5, characterized in that: The step of training the initial tea bud detection model based on the annotation information carried by the training tea bud picture and the prediction information to obtain a trained tea bud detection model specifically includes: Constructing a loss function term based on the annotation information carried by the training tea bud image and the prediction information; Training the initial tea bud detection model based on the loss function term to obtain a trained tea bud detection model; Wherein, the loss function term is expressed as: CIoU_Loss=1-IoU+αν+β(d / c)^2 Where v represents the correction factor, α and β represent adjustment coefficients, v represents the correction factor, d represents the Euclidean distance between the center point of the tender shoot annotation box in the annotation information and the center point of the prediction box in the prediction information, c represents the diagonal distance of the minimum circumscribed rectangle that contains both the prediction box and the tender shoot annotation box, IoU represents the intersection over union ratio of the annotation box and the prediction box, and w represents the intersection over union ratio of the annotation box and the prediction box. gt Indicates the width of the sprout annotation box in the annotation information, h gt represents the height of the sprout annotation box in the annotation information, w represents the width of the prediction box in the prediction information, and h represents the height of the prediction box in the prediction information.

8. A tea bud detection device based on deep learning, characterized in that: The tea bud detection device based on deep learning specifically includes: The collection module is used to collect tea buds pictures and build a training data set; A training module, used to train the initial tea bud detection model using the training data set to obtain a trained tea bud detection model; A control module is used to use the tea bud detection model to annotate the tea bud picture to automatically generate an annotation file, wherein the annotation file includes an annotation box and an annotation category, and the annotation category is one leaf and one bud or one leaf and two buds; the tea bud detection model adopts an improved yolov5 network structure, and an SE attention module is added to the backbone network of the improved yolov5 network structure, and the neck network of the improved yolov5 network structure includes two BiFPN structures, the front BiFPN structure is connected to the backbone network, and the rear BiFPN structure is connected to the head network of the yolov5 network structure.

Images