Tea leaf grading picking method, device, equipment and medium

Abstract

The application relates to a tea leaf grading picking method, device, equipment and medium, the method comprising: acquiring a tea leaf image to be picked in a tea garden; performing target detection on the tea leaf image to be picked based on a pre-trained tea leaf target detection model to determine a tea leaf target detection frame in the tea leaf image to be picked; performing key point detection on the tea leaf target detection frame based on a pre-trained tea leaf picking point detection model to determine the category of the tea leaf in the tea leaf image to be picked and the corresponding tea leaf coordinate information of the tea leaf, wherein the tea leaf category comprises one or any multiple of single-bud tea leaf, one-bud-one-leaf tea leaf and one-bud-two-leaf tea leaf; and a picking robot determines the corresponding picking points of the single-bud tea leaf, one-bud-one-leaf tea leaf and one-bud-two-leaf tea leaf according to the tea leaf category and the corresponding tea leaf coordinate information of the tea leaf to complete the grading picking of the tea leaf. The application can improve the completeness of the picked tea leaf and improve the accuracy and efficiency of the positioning of the tea leaf picking point.

Description

Technical Field

[0001] This application relates to the field of tea picking, and in particular to a method for grading and picking tea leaves, a corresponding device, electronic equipment, and a computer-readable storage medium. Background Technology

[0002] Tea is one of China's important agricultural products. The main methods for harvesting tea leaves are manual picking and mechanical picking. Manual picking is selective, resulting in high-quality tea. People can judge the suitability of buds for picking and their grade by observing their shape, color, and other characteristics. However, it is costly and inefficient. While using ordinary mechanical picking to replace manual picking can improve efficiency, mechanical picking lacks selectivity and may pick old leaves, tender buds, or even branches along with the leaves. Therefore, there is a need to research a highly accurate automated tea picking technology to replace both manual and ordinary mechanical picking. The key to this technology lies in the detection of tea leaves and the location of picking points.

[0003] In recent years, vision-based automated harvesting robots have been used for harvesting premium teas. However, the complex environment of tea gardens, with tea leaves shading each other, and excessively strong or dim lighting resulting in low resolution between tender buds and older leaves, makes the identification and location of tender bud harvesting points extremely difficult, severely limiting the automated harvesting of premium teas. Current methods for locating tea leaf harvesting points suffer from low accuracy and efficiency, and cannot achieve graded harvesting. To achieve rapid identification and location of harvesting points, implement scientific grading harvesting according to market demand and product processing needs, and ensure both the efficiency and high quality requirements of mechanized harvesting of premium teas, it is necessary to develop a method for acquiring the location information of premium tea harvesting points.

[0004] In summary, existing technologies suffer from several problems, including excessively strong or dim lighting, low resolution between tender buds and old leaves, which severely limit the automated harvesting of premium teas. Furthermore, current tea leaf picking location methods suffer from low accuracy and efficiency, and cannot achieve graded harvesting of tea leaves. Therefore, the applicant has made corresponding explorations to address these issues. Summary of the Invention

[0005] The purpose of this application is to solve the above-mentioned problems by providing a method for grading and harvesting tea leaves, a corresponding device, electronic equipment, and a computer-readable storage medium.

[0006] To achieve the various objectives of this application, the following technical solution is adopted:

[0007] A method for grading and harvesting tea leaves, proposed to meet one of the purposes of this application, includes:

[0008] Responding to the tea leaf grading and picking instructions, obtain images of tea leaves to be picked in the tea garden;

[0009] The tea leaf target detection model is used to detect targets in the image of tea leaves to be picked. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three feature maps are input into a multi-layer Transformer encoder for encoding. After being decoded by a multi-layer Transformer decoder, the tea leaf target detection box in the image of tea leaves to be picked is determined.

[0010] Based on the pre-trained tea leaf picking point detection model, key point detection is performed on the tea leaf target detection box to determine the category of tea leaves in the image of tea leaves to be picked and its corresponding tea leaf coordinate information. The tea leaf category includes one or any combination of single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves.

[0011] The picking robot determines the picking points corresponding to single-bud tea leaves, one-bud-one-leaf tea leaves, and one-bud-two-leaf tea leaves based on the tea leaf category and its corresponding coordinate information, so as to complete the grading and picking of tea leaves.

[0012] Optionally, the FcaBlock module includes convolutional layers, BatchNorm layers, ReLU activation functions, and a frequency domain attention mechanism.

[0013] Optionally, the SimSPPF module includes convolutional layers, BatchNorm layers, ReLU activation functions, and pooling layers.

[0014] Optionally, the backbone network of the tea leaf target detection model is a ResNest structure.

[0015] Optionally, the tea leaf picking point detection model includes a CoordConv module, a coordinate channel, a TransformerBlock module, and a polarized attention mechanism.

[0016] Optionally, the basic network architecture of the tea leaf target detection model is the DINO network model.

[0017] Optionally, the basic network architecture of the tea leaf picking point detection model is the HRNet network model.

[0018] A tea leaf grading and harvesting device provided for another purpose of this application includes:

[0019] The image acquisition module is configured to acquire images of tea leaves to be picked in the tea garden in response to tea leaf grading and picking instructions;

[0020] The tea leaf detection box determination module is configured to perform target detection on the image of tea leaves to be picked based on a pre-trained tea leaf target detection model. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three feature maps are input into a multi-layer Transformer encoder for encoding. After being decoded by a multi-layer Transformer decoder, the tea leaf target detection box in the image of tea leaves to be picked is determined.

[0021] The key point detection module is configured to perform key point detection on the tea leaf target detection box based on a pre-trained tea leaf picking point detection model, and determine the category of tea leaves in the image of tea leaves to be picked and its corresponding tea leaf coordinate information. The tea leaf category includes one or more of the following: single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves.

[0022] The tea leaf picking module is configured so that the picking robot determines the picking points corresponding to single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves based on the tea leaf category and its corresponding coordinate information, so as to complete the graded picking of tea leaves.

[0023] An electronic device provided for another purpose of this application includes a central processing unit and a memory, the central processing unit being used to invoke and run a computer program stored in the memory to perform the steps of the tea leaf grading and picking method described in this application.

[0024] A computer-readable storage medium is provided for another purpose of this application, which stores, in the form of computer-readable instructions, a computer program implemented according to the tea leaf grading and picking method, which, when called by a computer, executes the steps included in the corresponding method.

[0025] Compared to existing technologies, this application addresses the problems in existing technologies, such as excessively strong or dim lighting, low resolution between tender buds and old leaves, which severely limit the automated harvesting of premium teas, as well as the low accuracy and efficiency of current tea leaf harvesting point positioning methods, which cannot achieve graded harvesting of tea leaves. This application provides, but is not limited to, the following beneficial effects:

[0026] Firstly, the tea leaf grading and picking method of this application first uses a tea leaf target detection model to detect the tea leaves, and then uses a tea leaf picking point detection model to determine the picking points for single buds, one bud and one leaf, and one bud and two leaves based on the growth posture of the tea leaves. This achieves graded picking of tea leaves, improves the integrity of the picked tea leaves, reduces the influence of the surrounding environment on its positioning, and improves the accuracy and efficiency of tea leaf picking point positioning.

[0027] Secondly, the ResNest structure is used as the backbone network of the tea leaf target detection model. At the same time, a frequency domain attention module is added to the backbone network of the tea leaf target detection model to improve the network's ability to distinguish between tea leaves and old leaves, which can suppress the background and highlight the foreground target.

[0028] Third, the SimSPPF module is added to the last layer output of the backbone network of the tea leaf target detection model to achieve the fusion of features of different sizes, which is beneficial to solving the problem of large differences in target size in the detected image.

[0029] Fourth, the target detection and key point detection are combined. First, the tea leaf target detection model is used to detect the tender buds and some branches of the tea leaves. Then, the tea leaf picking point detection model is used to create a dense heat map of the detected tea leaves and predict the coordinate offset. Finally, the two are combined to obtain the coordinates of the key points.

[0030] Fifth, in the tea leaf picking point detection model, a CoordConv module is added before its first sub-network. As a task that is highly sensitive to positional information, key point detection can be enhanced by adding coordinates to the convolution, thus giving it spatial awareness. Furthermore, PSABlock is designed to extract image features, which can reduce the information loss caused by dimensionality reduction.

[0031] Sixth, the tea leaf picking point detection model can identify single bud picking points, one bud and one leaf picking points, and one bud and two leaves picking points, enabling graded picking of tea leaves and greatly improving the efficiency and quality of tea leaf picking. Attached Figure Description

[0032] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0033] Figure 1 This is a flowchart illustrating the tea leaf grading and harvesting method in the embodiments of this application;

[0034] Figure 2 This is a schematic diagram of the FcaBlock module in an embodiment of this application;

[0035] Figure 3 This is a schematic diagram of frequency domain attention in an embodiment of this application;

[0036] Figure 4 This is a schematic diagram of the SimSPPF module in an embodiment of this application;

[0037] Figure 5 This is a schematic diagram of the tea leaf target detection model in the embodiments of this application;

[0038] Figure 6This is a schematic diagram showing the coordinates of the tea leaf picking point obtained in the embodiments of this application;

[0039] Figure 7 This is a schematic diagram of the tea leaf picking point detection model in the embodiments of this application;

[0040] Figure 8 A schematic diagram illustrating the introduction of a multi-head attention mechanism into the TransformerBlock module in this application embodiment;

[0041] Figure 9 This is a schematic diagram of the PSABlock module in the tea leaf picking point detection model in the embodiments of this application;

[0042] Figure 10 This is a schematic diagram of the BottleNeck module in the tea leaf picking point detection model in the embodiments of this application;

[0043] Figure 11 This is a schematic diagram of the TransformerBlock module in the tea leaf picking point detection model in the embodiments of this application;

[0044] Figure 12 This is a schematic diagram of the tea leaf grading and picking device in the embodiments of this application;

[0045] Figure 13 This is a schematic diagram of the structure of the computer device in the embodiments of this application. Detailed Implementation

[0046] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0047] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0048] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0049] Those skilled in the art will understand that the terms "client," "terminal," and "terminal device" as used herein include both devices that receive wireless signals, devices that only possess wireless signal receiver capabilities without transmission capabilities, and devices with receiving and transmitting hardware, devices that have receiving and transmitting hardware capable of bidirectional communication over a bidirectional communication link. Such devices may include: cellular or other communication devices such as personal computers or tablets, having single-line displays, multi-line displays, or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service) that can combine voice, data processing, fax, and / or data communication capabilities; PDA (Personal Digital Assistant) that may include a radio frequency receiver, pager, internet / intranet access, web browser, notepad, calendar, and / or GPS (Global Positioning System) receiver; and conventional laptops and / or handheld computers or other devices that have and / or include radio frequency receivers. As used herein, "client," "terminal," and "terminal device" can be portable, transportable, installed in a means of transportation (air, sea, and / or land), or suitable and / or configured to operate locally and / or in a distributed manner, operating in any other location on Earth and / or in space. "Client," "terminal," and "terminal device" as used herein can also be a communication terminal, an internet access terminal, or a music / video playback terminal, such as a PDA, a MID (Mobile Internet Device), and / or a mobile phone with music / video playback capabilities, or a smart TV, set-top box, etc.

[0050] The hardware referred to by the names "server," "client," and "service node" in this application is essentially an electronic device with the equivalent capabilities of a personal computer. It is a hardware device with the necessary components revealed by the von Neumann architecture, such as a central processing unit (including an arithmetic logic unit and a control unit), memory, input devices, and output devices. The computer program is stored in its memory, and the central processing unit loads the program stored in the secondary storage into the main memory to run it, execute the instructions in the program, and interact with the input and output devices to complete specific functions.

[0051] It should be noted that the concept of "server" used in this application can also be extended to the case of server clusters. Based on the network deployment principles understood by those skilled in the art, the servers should be logically divided. Physically, these servers can be independent of each other but accessible through interfaces, or they can be integrated into a single physical computer or a computer cluster. Those skilled in the art should understand this flexibility and should not use it to constrain the implementation of the network deployment method in this application.

[0052] One or more of the technical features of this application, unless explicitly specified herein, can be deployed on a server and accessed by a client remotely calling the online service interface provided by the server, or can be directly deployed and run on a client for access.

[0053] Unless otherwise specified, the neural network models referenced or potentially referenced in this application may be deployed on a remote server and invoked remotely on the client, or deployed on a client with the capability to invoke directly. In some embodiments, when running on the client, the corresponding intelligence may be acquired through transfer learning in order to reduce the requirements on the client's hardware resources and avoid excessive consumption of the client's hardware resources.

[0054] Unless otherwise specified, all data involved in this application may be stored remotely on a server or on a local terminal device, as long as it is suitable for use by the technical solution of this application.

[0055] Those skilled in the art will understand that although the various methods in this application are described based on the same concept and thus present commonality among them, they can be performed independently unless otherwise specified. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept; therefore, concepts expressed in the same way, as well as concepts that are appropriately changed for convenience but are expressed differently, should be understood equivalently.

[0056] Unless otherwise expressly stated, the various embodiments disclosed in this application can be combined in a cross-cutting manner to flexibly construct new embodiments, as long as such combination does not depart from the inventive spirit of this application and can meet the needs of the prior art or solve a certain deficiency in the prior art. Those skilled in the art should be aware of such modifications.

[0057] Based on the above exemplary scenarios, please refer to Figure 1 In one embodiment of the tea leaf grading and harvesting method of this application, the method includes:

[0058] Step S10: Respond to the tea leaf grading and picking instruction, and obtain images of tea leaves to be picked in the tea garden;

[0059] The tea-picking robot in the tea garden can respond to the tea leaf grading and picking instructions and acquire images of tea leaves to be picked in the tea garden. The images of tea leaves to be picked are images of tea leaves containing single buds, one bud and one leaf, or one bud and two leaves. The images of tea leaves to be picked can be acquired by the binocular camera in the picking robot.

[0060] Step S20: Based on the pre-trained tea leaf target detection model, target detection is performed on the image of tea leaves to be picked. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three feature maps are input into the multi-layer Transformer encoder for encoding. After being decoded by the multi-layer Transformer decoder, the tea leaf target detection box in the image of tea leaves to be picked is determined.

[0061] The basic network architecture of the tea leaf target detection model is an improved DINO network model. The DINO network model includes a backbone module, a multi-layer Transformer encoder, a multi-layer Transformer decoder, and multiple detection heads. The backbone network of the tea leaf target detection model is a ResNest structure.

[0062] The backbone network of the tea leaf target detection model is responsible for extracting the feature representation of the input image. This backbone network can extract three layers of feature maps. In the tea leaf target detection model of this application, the improved ResNest structure is used as the backbone network of the tea leaf target detection model. The improved ResNest structure introduces the FcaBlock module and SimSPPF module based on frequency domain attention on the basis of ResNest. The output feature map of the last layer of the backbone network of the tea leaf target detection model is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the second-to-last and third-to-last layers of the backbone network of the tea leaf target detection model are processed by frequency domain attention. The output of the last three layers of feature maps is input into the multi-layer Transformer encoder for encoding.

[0063] like Figure 2 As shown, the FcaBlock module consists of convolutional layers, BatchNorm layers, ReLU activation functions, and a frequency domain attention mechanism. After the data passes through the convolutional layers, the height and width of the input image are halved, while the channel depth is doubled. Figure 3 As shown, frequency domain attention treats the feature channel representation problem as a compression process using frequency analysis. By introducing more frequency components, it makes full use of information, thereby strengthening important information and suppressing unimportant information.

[0064] like Figure 4 As shown, the SimSPPF module comes from YOLOv6. This module consists of convolutional layers, BatchNorm layers, ReLU activation functions, and pooling layers. This module effectively avoids image distortion caused by cropping and scaling operations on image regions, and solves the problem of extracting graph-related repetitive features in convolutional neural networks.

[0065] like Figure 5 As shown, the multi-layer Transformer encoder gradually extracts a high-level abstract representation of the features by processing the input features in multiple layers, and encodes the image into a set of feature vectors. The multi-layer Transformer encoder consists of several Transformer blocks, and each Transformer block consists of two sub-layers, namely a self-attention layer and a feed-forward neural network layer.

[0066] like Figure 5 As shown, the multi-layer Transformer decoder is mainly used for the implementation of the decoder. It decodes the information obtained by the encoder and the predicted position, category and other information to obtain the final target detection result and determine the target detection box of tea leaves in the image of tea leaves to be picked.

[0067] In some embodiments, the improved DINO network is trained, and training parameters are set to train the tea leaf target detection model, specifically including:

[0068] During training, the Adam W optimizer was used, with an initial learning rate of 0.0001, a weight decay of 0.0001, FocalLoss as the classification loss function, L1Loss as the object detection box loss function, 50 epochs, and a batch size of 8.

[0069] The training set of the tea leaf dataset is input into the model for training, and the training is performed according to the set parameters. The network parameters are updated once every epoch. The validation set data is input into the model for prediction, and the model accuracy is evaluated. The model parameters are saved in the file after every 10 epochs of training, and the model parameters with the highest validation accuracy are selected as the optimal solution. Finally, the tea leaf recognition model is obtained.

[0070] Use PyTorch to load the model file parameters and construct the network model. Input the tea leaf image to be identified into the trained network model, and the model outputs the target detection box of the tea leaf to be identified.

[0071] In some embodiments, tea leaf images are acquired using an image acquisition device in various ways, including shooting from a vertical overhead angle, at a 45° or 30° angle to the ground, or shooting from the side perpendicular to the ground. Specifically, images of tea leaves can be acquired using an image acquisition device at a 45° or 30° angle to the ground. Rectangular bounding boxes are then labeled on the tea leaf images to construct a tea leaf dataset. The labeled tea leaf images are then divided into a training set and a validation set at a 9:1 ratio.

[0072] Step S30: Based on the pre-trained tea leaf picking point detection model, perform key point detection on the tea leaf target detection box to determine the category of tea leaves in the tea leaf image to be picked and its corresponding tea leaf coordinate information. The tea leaf category includes one or more of the following: single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves.

[0073] The basic network architecture of the tea leaf picking point detection model is an improved HRNet network model. After determining the target bounding box of the tea leaves in the image of the tea leaves to be picked, the model performs key point detection on the target bounding box based on the pre-trained tea leaf picking point detection model to determine the category of the tea leaves in the image of the tea leaves to be picked and its corresponding coordinate information. The category of tea leaves includes one or more of the following: single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves. Figure 6As shown, the tea leaf image to be identified is first input into the trained tea leaf target detection model to obtain the tea leaf target detection box. Then, the tea leaf target detection box is input into the key point detection model to determine the category of tea leaves in the tea leaf image to be picked and its corresponding tea leaf coordinate information. Finally, the coordinates of the tea leaf picking point are obtained.

[0074] In the HRNet network model, the tea leaf picking point detection model includes a CoordConv module, a coordinate channel, a TransformerBlock module, and a polarization attention mechanism. First, a high-resolution sub-network is used as the first stage, and then sub-networks from high resolution to low resolution are gradually added, resulting in outputs from multiple stages. Finally, the outputs of the multi-resolution sub-networks are connected in parallel. This multi-scale fusion ensures that each high-resolution to low-resolution feature map representation can receive information from other parallel representation branches repeatedly, thus obtaining a more information-rich high-resolution representation.

[0075] like Figure 7 As shown, a CoordConv module is added before the first sub-network. As a task that is highly sensitive to positional information, keypoint detection can add coordinates to the convolution, thus giving it spatial awareness.

[0076] like Figure 7 As shown, a TransformerBlock is added to the fourth sub-network, allowing HRNet to combine the advantages of both convolution and Transformer. Convolution can efficiently learn abstract low-resolution feature maps from large images. The Transformer is used to process and aggregate the information contained in the feature maps captured by the convolution. The TransformerBlock comes from BotNet. Figure 8 As shown, TransformerBlock introduces a multi-head self-attention mechanism on top of ResNet's BottleNeck module.

[0077] like Figure 9 As shown, two coordinate channels are added to the input feature map to represent the x and y coordinates of the original input, respectively, before traditional convolution is performed. This allows the convolution process to be aware of the spatial information of the feature map.

[0078] like Figure 10As shown, PSABlock is designed based on the polarized self-attention mechanism. Polarized attention can reduce the information loss caused by dimensionality reduction. Fine-grained information is very important for keypoint localization, so the addition of polarized self-attention greatly improves the accuracy of the model.

[0079] In some embodiments, Figure 11 This is a schematic diagram of the TransformerBlock module in the HRNet network model.

[0080] In some embodiments, the improved HRNet network is trained, and training parameters are set to train the tea leaf picking point detection model, specifically including:

[0081] The Adam optimizer was used during training, with an initial learning rate of 0.0001, weight decay of 0.05, and 300 epochs.

[0082] The network parameters are updated once per epoch. Validation set data is input into the model for prediction, and the model accuracy is evaluated. After every 10 epochs of training, the model file parameters are saved, and the model parameters with the highest validation accuracy are selected as the optimal solution. The model file parameters are loaded using PyTorch, and a tea leaf picking point detection model is constructed.

[0083] Step S40: The picking robot determines the picking points corresponding to the single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves according to the tea leaf category and its corresponding tea leaf coordinate information, so as to complete the grading and picking of tea leaves.

[0084] After determining the category of tea leaves in the image of tea leaves to be picked and their corresponding coordinate information, the picking robot determines the picking points corresponding to single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves according to the category of tea leaves and their corresponding coordinate information, so as to complete the graded picking of tea leaves.

[0085] As can be seen from the above embodiments, compared with the prior art, this application addresses the problems in the prior art, such as excessively strong or dim lighting, low resolution between tender buds and old leaves, which severely limit the automated harvesting of premium teas, as well as the low positioning accuracy and efficiency of the current tea leaf harvesting point positioning methods, and the inability to achieve graded harvesting of tea leaves. This application has, but is not limited to, the following beneficial effects:

[0086] Firstly, the tea leaf grading and picking method of this application first uses a tea leaf target detection model to detect the tea leaves, and then uses a tea leaf picking point detection model to determine the picking points for single buds, one bud and one leaf, and one bud and two leaves based on the growth posture of the tea leaves. This achieves graded picking of tea leaves, improves the integrity of the picked tea leaves, reduces the influence of the surrounding environment on its positioning, and improves the accuracy and efficiency of tea leaf picking point positioning.

[0087] Secondly, the ResNest structure is used as the backbone network of the tea leaf target detection model. At the same time, a frequency domain attention module is added to the backbone network of the tea leaf target detection model to improve the network's ability to distinguish between tea leaves and old leaves, which can suppress the background and highlight the foreground target.

[0088] Third, the SimSPPF module is added to the last layer output of the backbone network of the tea leaf target detection model to achieve the fusion of features of different sizes, which is beneficial to solving the problem of large differences in target size in the detected image.

[0089] Fourth, the target detection and key point detection are combined. First, the tea leaf target detection model is used to detect the tender buds and some branches of the tea leaves. Then, the tea leaf picking point detection model is used to create a dense heat map of the detected tea leaves and predict the coordinate offset. Finally, the two are combined to obtain the coordinates of the key points.

[0090] Fifth, in the tea leaf picking point detection model, a CoordConv module is added before its first sub-network. As a task that is highly sensitive to positional information, key point detection can be enhanced by adding coordinates to the convolution, thus giving it spatial awareness. Furthermore, PSABlock is designed to extract image features, which can reduce the information loss caused by dimensionality reduction.

[0091] Sixth, the tea leaf picking point detection model can identify single bud picking points, one bud and one leaf picking points, and one bud and two leaves picking points, enabling graded picking of tea leaves and greatly improving the efficiency and quality of tea leaf picking.

[0092] Please see Figure 12This application provides a tea leaf grading and harvesting device, comprising an image acquisition module 1100, a tea leaf detection box determination module 1200, a key point detection module 1300, and a tea leaf harvesting module 1400. The image acquisition module 1100 is configured to acquire images of tea leaves to be harvested in a tea garden in response to a tea leaf grading and harvesting instruction. The tea leaf detection box determination module 1200 is configured to perform target detection on the images of tea leaves to be harvested based on a pre-trained tea leaf target detection model. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by an FcaBlock module and a SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three layers of feature maps are input into a multi-layer Transformer encoder for encoding. After decoding by the ORMER decoder, the target detection box of tea leaves in the image of tea leaves to be picked is determined; the key point detection module 1300 is set to perform key point detection on the target detection box of tea leaves based on the pre-trained tea leaf picking point detection model, and determine the category of tea leaves in the image of tea leaves to be picked and its corresponding coordinate information. The category of tea leaves includes one or more of the following: single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves; the tea leaf picking module 1400 is set to determine the picking point corresponding to the single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves according to the category of tea leaves and its corresponding coordinate information, so as to complete the graded picking of tea leaves.

[0093] Based on any embodiment of this application, please refer to Figure 13 Another embodiment of this application also provides an electronic device, which can be implemented by a computer device, such as... Figure 13 The diagram shows the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. The computer-readable storage medium stores an operating system, a database, and computer-readable instructions. The database may store a sequence of control information. When the computer-readable instructions are executed by the processor, they enable the processor to implement a tea leaf grading and harvesting method. The processor provides computing and control capabilities to support the operation of the entire computer device. The memory stores computer-readable instructions, which, when executed by the processor, enable the processor to execute the tea leaf grading and harvesting method of this application. The network interface of the computer device is used for communication with a terminal. Those skilled in the art will understand that… Figure 13The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0094] In this embodiment, the processor is used to execute... Figure 12 The system contains the specific functions of each module and its sub-modules, and the memory stores the program code and various data required to execute these modules or sub-modules. The network interface is used for data transmission between the user terminal and the server. In this embodiment, the memory stores the program code and data required to execute all modules / sub-modules in the tea leaf grading and harvesting device of this application, and the server can call the server's program code and data to execute the functions of all sub-modules.

[0095] This application also provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the tea leaf grading and picking method described in any embodiment of this application.

[0096] This application also provides a computer program product, including a computer program / instructions that, when executed by one or more processors, implement the steps of the tea leaf grading and picking method described in any embodiment of this application.

[0097] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0098] The above description is only a partial embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

[0099] In summary, the tea leaf grading and picking method of this application first uses a tea leaf target detection model to detect tea leaves, and then uses a tea leaf picking point detection model to determine the picking points for single buds, one bud and one leaf, and one bud and two leaves based on the growth posture of the tea leaves. This achieves graded picking of tea leaves, improves the integrity of the picked tea leaves, reduces the influence of the surrounding environment on its positioning, and improves the accuracy and efficiency of tea leaf picking point positioning.

Claims

1. A method for grading and harvesting tea leaves, characterized in that, include: Responding to the tea leaf grading and picking instructions, obtain images of tea leaves to be picked in the tea garden; The tea leaf target detection model is used to detect targets in the image of tea leaves to be picked. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three feature maps are input into a multi-layer Transformer encoder for encoding. After decoding by a multi-layer Transformer decoder, the tea leaf target detection box in the image of tea leaves to be picked is determined. The basic network architecture of the tea leaf target detection model is the DINO network model, and the backbone network of the tea leaf target detection model is a ResNest structure. Based on the pre-trained tea leaf picking point detection model, key point detection is performed on the tea leaf target detection box to determine the category of tea leaves in the image of tea leaves to be picked and its corresponding tea leaf coordinate information. The tea leaf category includes one or any combination of single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves. The picking robot determines the picking points corresponding to single-bud tea leaves, one-bud-one-leaf tea leaves, and one-bud-two-leaf tea leaves based on the tea leaf category and its corresponding coordinate information, so as to complete the grading and picking of tea leaves.

2. The method for grading and harvesting tea leaves according to claim 1, characterized in that, The FcaBlock module includes convolutional layers, BatchNorm layers, ReLU activation functions, and a frequency domain attention mechanism.

3. The method for grading and harvesting tea leaves according to claim 1, characterized in that, The SimSPPF module includes convolutional layers, BatchNorm layers, ReLU activation functions, and pooling layers.

4. The method for grading and harvesting tea leaves according to claim 1, characterized in that, The tea leaf picking point detection model includes a CoordConv module, a coordinate channel, a TransformerBlock module, and a polarized attention mechanism.

5. The method for grading and harvesting tea leaves according to claim 1, characterized in that, The basic network architecture of the tea leaf picking point detection model is the HRNet network model.

6. A tea leaf grading and picking device, characterized in that, include: The image acquisition module is configured to acquire images of tea leaves to be picked in the tea garden in response to tea leaf grading and picking instructions; The tea leaf target detection box determination module is configured to perform target detection on the image of tea leaves to be picked based on a pre-trained tea leaf target detection model. In the backbone network of the tea leaf target detection model, the output feature map of the last layer of the backbone network is processed by the FcaBlock module and the SimSPPF module. The output feature maps of the penultimate and penultimate layers of the backbone network are processed by frequency domain attention. The last three layer feature maps are input into a multi-layer Transformer encoder for encoding. After decoding by a multi-layer Transformer decoder, the tea leaf target detection box in the image of tea leaves to be picked is determined. The basic network architecture of the tea leaf target detection model is the DINO network model, and the backbone network of the tea leaf target detection model is a ResNest structure. The key point detection module is configured to perform key point detection on the tea leaf target detection box based on a pre-trained tea leaf picking point detection model, and determine the category of tea leaves in the image of tea leaves to be picked and its corresponding tea leaf coordinate information. The tea leaf category includes one or more of the following: single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves. The tea leaf picking module is configured so that the picking robot determines the picking points corresponding to single bud tea leaves, one bud and one leaf tea leaves, and one bud and two leaves tea leaves based on the tea leaf category and its corresponding coordinate information, so as to complete the graded picking of tea leaves.

7. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implemented according to any one of claims 1 to 5, which, when invoked by a computer, executes the steps included in the corresponding method.

Images