A digital in-situ yield estimation method and device for yinghong No. 9 tea garden

Abstract

This invention relates to the field of artificial intelligence technology, specifically a method and apparatus for digital in-situ yield estimation of Yinghong No. 9 tea gardens. The method includes: matching the correspondence of tea shoots between images of various tea gardens to obtain matching relationships; calculating the viewpoint-position relationship of each tea garden image based on the matching relationships to obtain a set of viewpoint-position relationships; performing modeling and reconstruction operations on a set of shoot objects based on the set of viewpoint-position relationships to obtain a set of shoot object models; obtaining key information sequences for each shoot object model in the set of shoot object models to obtain a set of key information sequences; and calculating the weight composition sequence of each shoot object model based on a yield conversion empirical model and the set of key information sequences to obtain a set of weight composition sequences and perform tea yield estimation operations to obtain the estimated tea yield. This invention can improve the accuracy and efficiency of tea yield estimation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a digital in-situ yield estimation method and apparatus for Yinghong No. 9 tea gardens. Background Technology

[0002] Accurate yield forecasts for tea gardens can provide a basis for tea farmers to adjust their irrigation, fertilization, and harvesting plans. At the same time, the yield estimates within the region can predict market price trends, help government departments formulate reasonable industrial policies, guide tea farmers to set reasonable prices or adjust their sales strategies, and avoid income losses due to oversupply.

[0003] The yield of a tea garden is determined by the weight of the tender tea shoots picked. Depending on the picking standards of Yinghong No. 9, it can be divided into single bud, one bud and one leaf, and one bud and two leaves. However, the yield of Yinghong No. 9 tea garden is currently mainly calculated by weighing the actual tea leaves picked, making it difficult to arrange production activities by estimating the yield in situ. Summary of the Invention

[0004] This invention provides a digital in-situ yield estimation method for Yinghong No. 9 tea gardens, the main purpose of which is to improve the accuracy and efficiency of tea yield estimation.

[0005] To achieve the above objectives, this invention provides a digital in-situ yield estimation method for Yinghong No. 9 tea gardens, comprising:

[0006] A sequence of tea garden images is obtained, and the pluckable tender shoots in each tea garden image in the sequence are identified to obtain a set of tender shoot object sequences.

[0007] Using a pre-built deep learning feature extraction network, tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set are extracted from the tea garden image sequence to obtain a tender shoot fingerprint feature sequence set.

[0008] Using a pre-constructed deep feature matching network, the corresponding relationship of tea tree shoots between tea garden images in the tea garden image sequence is matched according to the set of tender shoot fingerprint feature sequences, so as to obtain the tea tree shoot matching relationship;

[0009] Using pre-built camera pose estimation software, the viewpoint-position relationship of each tea garden image in the tea garden image sequence is calculated based on the matching relationship of the tea tree shoots, resulting in a viewpoint-position relationship set. Based on the viewpoint-position relationship set, the shoot object sequence set is modeled and reconstructed to obtain a shoot object model set. The key information sequence of each shoot object model in the shoot object model set is then obtained to obtain a key information sequence set.

[0010] Based on the pre-constructed yield conversion experience model and the set of key information sequences, the weight composition sequence of each tender shoot object model in the set of tender shoot object models is calculated to obtain the set of weight composition sequences.

[0011] Based on the pre-constructed seasonal influence parameters, the tea yield is estimated by performing a tea yield estimation operation on the set of weight composition sequences.

[0012] Optionally, the step of acquiring a tea garden image sequence and identifying pluckable tender shoots in each tea garden image within the tea garden image sequence to obtain a set of tender shoot object sequences includes:

[0013] Acquire videos of tender tea shoots per unit area of ​​tea garden during the harvest season, and perform frame extraction processing on the videos of tender tea shoots to obtain a tea garden image sequence;

[0014] Using a pre-constructed target detection network, feature extraction is performed on each tea garden image in the tea garden image sequence to obtain a tea image feature sequence set.

[0015] The tea image feature sequence set is subjected to a pickable tender shoot identification operation to obtain a tender shoot object sequence set.

[0016] Optionally, the step of using a pre-constructed target detection network to perform feature extraction operations on each tea garden image in the tea garden image sequence to obtain a tea leaf image feature sequence set includes:

[0017] One tea garden image is obtained sequentially from the tea garden image sequence to obtain the target tea garden image;

[0018] The target tea garden image is subjected to multi-resolution feature extraction using the pre-built ResNet18 network in the pre-built target detection network to obtain an initial multi-layer convolutional feature map, and a residual connection operation is performed on the initial multi-layer convolutional feature map to obtain a multi-layer feature map.

[0019] Using a pre-built GD-Tea feature fusion network, low-high semantic information is fused into the multi-layer feature map to obtain a semantically fused multi-layer feature map, and the semantically fused multi-layer feature map is then spliced ​​to obtain a multi-scale feature map.

[0020] The multi-scale feature map is segmented using a pre-constructed RDE-Head network to obtain the tea leaf image feature sequence of the target tea garden image.

[0021] The tea leaf image feature sequences corresponding to each target tea garden image in the tea garden image sequence are summarized to obtain a tea leaf image feature sequence set.

[0022] Optionally, the step of using a pre-constructed GD-Tea feature fusion network to perform low-to-high semantic information fusion on the multi-layer feature map to obtain a semantically fused multi-layer feature map, and then performing feature concatenation on the semantically fused multi-layer feature map to obtain a multi-scale feature map, including:

[0023] Using the pre-built cascaded group attention module in the pre-built GD-Tea feature fusion network, the target deep feature map in the multi-layer feature map is subjected to cascaded group attention enhancement operation to obtain the target deep enhanced feature map;

[0024] Using the low-order guidance module pre-built in the GD-Tea feature fusion network, target shallow feature extraction is performed on the multi-layer feature map to obtain shallow features;

[0025] Using the shallow features, a cross-scale connection is performed on the preset first target layer feature map in the multi-layer feature map to obtain a shallow feature fusion multi-layer feature map;

[0026] Using a pre-built feature enhancement module, feature enhancement operations are performed on the shallow feature fusion multi-layer feature map to obtain a shallow feature enhanced multi-layer feature map;

[0027] Using the high-order guidance module pre-built in the GD-Tea feature fusion network, target deep feature extraction operation is performed on the shallow feature enhancement multi-layer feature map and the target deep enhancement feature map to obtain deep features;

[0028] Using the deep features, cross-scale connections are made between the second target layer feature map in the shallow feature enhancement multi-layer feature map to obtain a deep feature fusion feature map, and cross-scale connections are made between the target deep enhancement feature map to obtain a target deep fusion feature map.

[0029] Using the feature enhancement module, feature enhancement operations are performed on the deep feature fusion feature map to obtain a deep feature enhanced feature map, and feature enhancement operations are performed on the target deep fusion feature map to obtain a target deep fusion enhanced feature map;

[0030] The feature maps of the deep feature enhancement map, the target deep fusion enhancement map, and the third target layer feature map in the shallow feature enhancement multi-layer feature map are spliced ​​together to obtain a multi-scale feature map.

[0031] Optionally, the step of using the feature enhancement module to perform feature enhancement operations on the deep feature fusion feature map to obtain a deep feature enhanced feature map includes:

[0032] Using the feature enhancement module, a dilated convolution operation based on a preset number of branches is performed on the deep feature fusion feature map according to a preset set of dilated convolution parameters to obtain a dilated convolution feature map sequence.

[0033] Batch normalization is performed on each dilated convolutional feature map in the dilated convolutional feature map sequence to obtain a normalized dilated convolutional feature map sequence.

[0034] A feature fusion operation is performed on each standardized dilated convolutional feature map in the standardized dilated convolutional feature map sequence to obtain a deep feature enhancement feature map.

[0035] Optionally, the step of performing a pickable tender shoot identification operation on the tea image feature sequence set to obtain a tender shoot object sequence set includes:

[0036] The tea leaf image feature sequence set is subjected to tender shoot contour recognition to obtain a set of tender shoot bounding boxes;

[0037] Based on the pre-constructed Kalman filter algorithm, the motion state of each tender shoot outer frame in the tender shoot outer frame set is monitored to obtain a tracker set;

[0038] Based on the pre-constructed Hungarian matching association algorithm, the set of trackers is used to perform a deduplication operation on the set of tender shoot bounding boxes to obtain a set of tender shoot object sequences.

[0039] Optionally, the step of performing a modeling and reconstruction operation on the set of tender shoot object sequences based on the viewpoint-position relationship set to obtain a set of tender shoot object models includes:

[0040] Using the pre-built position encoder in the camera pose estimation software, the viewpoint-position relationship set is encoded based on mapping to three-dimensional space to obtain a three-dimensional position encoding vector;

[0041] Using a pre-constructed two-layer multilayer perceptron, a radiation field attribute prediction operation is performed on the three-dimensional position encoding vector to obtain a three-dimensional representation set of tender shoots;

[0042] Based on the pre-constructed CtF sampling strategy, the three-dimensional representation set of the tender shoots is modeled to obtain a set of tender shoot object models.

[0043] Optionally, before the step of basing the method on the pre-constructed empirical model for output conversion and the set of key information sequences, the method further includes:

[0044] Obtain tea leaf sample data;

[0045] The target parameter relationship regression fitting operation was performed on the tea leaf sample data to obtain the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight.

[0046] Models were constructed for the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight to obtain an empirical model for yield conversion.

[0047] Optionally, before basing the steps on pre-constructed seasonal influence parameters, the method further includes:

[0048] Acquire tea leaf data samples from different seasons, configure initial seasonal parameters for the yield conversion empirical model, and obtain the seasonally initialized yield conversion empirical model.

[0049] Using the aforementioned seasonal initialization yield conversion empirical model, predictions are made on tea leaf data samples from different seasons to obtain a predicted tea leaf weight sequence.

[0050] Calculate the loss value between the pre-constructed real tea leaf weight sequence and the predicted tea leaf weight sequence in the tea leaf data samples of different seasons;

[0051] Minimize the loss value to obtain the network model parameters with the minimum loss value, and use the network model parameters to update the initial seasonal parameters to obtain the seasonal influence parameters.

[0052] To achieve the above objectives, the present invention also provides a digital in-situ yield estimation device for Yinghong No. 9 tea gardens, comprising:

[0053] The target detection module is used to acquire a sequence of tea garden images and identify the pluckable tender shoots in each tea garden image in the sequence of tea garden images to obtain a set of tender shoot object sequences;

[0054] The tea leaf modeling module is used to extract tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set from the tea garden image sequence using a pre-built deep learning feature extraction network, to obtain a tender shoot fingerprint feature sequence set; and to perform correspondence matching of tea tree tender shoots between tea garden images in the tea garden image sequence based on the tender shoot fingerprint feature sequence set, to obtain tea tree tender shoot matching relationships; and to calculate the viewpoint-position relationship of each tea garden image in the tea garden image sequence based on the tea tree tender shoot matching relationship using pre-built camera pose estimation software, to obtain a viewpoint-position relationship set; and to perform modeling and reconstruction operations on the tender shoot object sequence set based on the viewpoint-position relationship set, to obtain a tender shoot object model set; and to obtain the key information sequence of each tender shoot object model in the tender shoot object model set, to obtain a key information sequence set.

[0055] The weight calculation module is used to calculate the weight composition sequence of each tender shoot object model in the tender shoot object model set based on the pre-constructed yield conversion experience model and the key information sequence set, so as to obtain the weight composition sequence set.

[0056] The seasonal adjustment module is used to perform tea yield estimation on the set of weight composition sequences based on pre-constructed seasonal influence parameters to obtain the estimated tea yield.

[0057] To address the above problems, the present invention also provides an electronic device, the electronic device comprising:

[0058] Memory, storing at least one instruction;

[0059] The processor executes the instructions stored in the memory to implement the above-described digital in-situ yield estimation method for Yinghong No. 9 tea gardens.

[0060] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the aforementioned digital in-situ yield estimation method for Yinghong No. 9 tea gardens.

[0061] To address the problems described in the background art, this invention first identifies the tender shoot objects and their quantity in tea garden images through object detection. Then, it uses Colmap to perform refined modeling of the tender shoot objects, thereby estimating the weight of the tender shoots. However, to adapt to situations where tea leaves are small, numerous, and the characteristics of the tender shoots are similar to those of surrounding tea leaves, this invention improves the modeling process by employing an RT-DETR detection model based on the Transformer framework. It replaces the multi-head self-attention (MHSA) mechanism in the AIFI module with cascaded grouped attention (CGFA). Through slicing and cascading operations, each attention head acquires local information... Simultaneously, all information from the preceding header is obtained, deep features are optimized, semantic information is enriched, and computational redundancy is reduced. Furthermore, the GD-Tea feature fusion mechanism effectively integrates low-level and high-level semantic information from tea bud images, ensuring the model can accurately identify tea buds in complex environments. This invention also combines the Hungarian matching algorithm and the Kalman filter tracking algorithm to accurately count the harvestable tea shoots in tea garden image sequences from video. Additionally, this invention adaptively improves Colmap by replacing the traditional SIFT with a deep learning feature extraction network and the ANN matching module with a deep feature matching network, thereby improving the accuracy of shoot modeling. Therefore, this invention can improve the accuracy and efficiency of tea yield estimation. Attached Figure Description

[0062] Figure 1 This is a flowchart illustrating a digital in-situ yield estimation method for Yinghong No. 9 tea gardens provided in an embodiment of the present invention.

[0063] Figure 2 This is a functional module diagram of a digital in-situ yield estimation device for Yinghong No. 9 tea gardens provided in an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of the structure of an electronic device for implementing the digital in-situ yield estimation method for Yinghong No. 9 tea gardens, according to an embodiment of the present invention.

[0065] Explanation of reference numerals in the attached figures:

[0066] 10. Electronic device; 11. Processor; 12. Memory; 13. Bus.

[0067] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0068] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0069] This application provides a digital in-situ yield estimation method for Yinghong No. 9 tea gardens. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the digital in-situ yield estimation method for Yinghong No. 9 tea gardens can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0070] Reference Figure 1 The diagram shown is a flowchart illustrating a digital in-situ yield estimation method for Yinghong No. 9 tea gardens according to an embodiment of the present invention. In this embodiment, the digital in-situ yield estimation method for Yinghong No. 9 tea gardens includes:

[0071] S1. Obtain a sequence of tea garden images and identify the pluckable tender shoots in each tea garden image in the sequence to obtain a set of tender shoot object sequences.

[0072] The tea garden image sequence refers to a series of tea garden images extracted from tea garden videos. Tea garden videos refer to video data captured by devices such as drones on a specific area of ​​tea gardens.

[0073] The term "harvestable tender shoots" refers to tender shoots as specified in the tea harvesting standards of this invention. The tea harvesting standards of this invention are defined as those for Yinghong No. 9 (the main variety of Yingde black tea).

[0074] The set of tender shoot object sequences refers to the harvestable tender shoots identified in the tea garden image sequence.

[0075] In detail, in this embodiment of the invention, the step of acquiring a tea garden image sequence and identifying pluckable tender shoots in each tea garden image within the tea garden image sequence to obtain a set of tender shoot object sequences includes:

[0076] Acquire videos of tender tea shoots per unit area of ​​tea garden during the harvest season, and perform frame extraction processing on the videos of tender tea shoots to obtain a tea garden image sequence;

[0077] Using a pre-constructed target detection network, feature extraction is performed on each tea garden image in the tea garden image sequence to obtain a tea image feature sequence set.

[0078] The tea image feature sequence set is subjected to a pickable tender shoot identification operation to obtain a tender shoot object sequence set.

[0079] The tea garden per unit area during the harvesting period refers to the tea plantation per unit area during the period when the tea leaves can be harvested, with approximately eight harvesting seasons per year.

[0080] The video of the tender tea shoots is the same as the tea garden video mentioned above.

[0081] The frame extraction process refers to the extraction of images corresponding to certain frames in a video of tea shoots, for example, acquiring a tea garden image every 0.5 seconds. The tea garden image sequence refers to a collection of tea garden images obtained through the frame extraction process.

[0082] The target detection network refers to a neural network used to detect target objects.

[0083] The feature extraction operation refers to the process of extracting and reducing the dimensionality of data features from the tea garden image sequence. The tea leaf image feature sequence set refers to the feature extraction results of the tea garden image sequence.

[0084] The picking tender shoot identification operation refers to the operation of automatically selecting and marking picking tender shoots in the tea image feature sequence set through a neural network.

[0085] Specifically, in this embodiment of the invention, a drone is used to capture video of a unit area of ​​tea garden during the harvesting season. Then, the captured tea garden video is processed by frame extraction to obtain a tea garden image sequence. Then, a pre-constructed target detection network is used to perform feature extraction operations on each tea garden image to obtain a set of tea leaf image feature sequences. Finally, through the identification of pluckable tender shoots, a set of tender shoot object sequences and a set of tender shoot quantities are obtained.

[0086] In detail, in this embodiment of the invention, the step of using a pre-constructed target detection network to perform feature extraction operations on each tea garden image in the tea garden image sequence to obtain a tea leaf image feature sequence set includes:

[0087] One tea garden image is obtained sequentially from the tea garden image sequence to obtain the target tea garden image;

[0088] The target tea garden image is subjected to multi-resolution feature extraction using the pre-built ResNet18 network in the pre-built target detection network to obtain an initial multi-layer convolutional feature map, and a residual connection operation is performed on the initial multi-layer convolutional feature map to obtain a multi-layer feature map.

[0089] Using a pre-built GD-Tea feature fusion network, low-high semantic information is fused into the multi-layer feature map to obtain a semantically fused multi-layer feature map, and the semantically fused multi-layer feature map is then spliced ​​to obtain a multi-scale feature map.

[0090] The multi-scale feature map is segmented using a pre-constructed RDE-Head network to obtain the tea leaf image feature sequence of the target tea garden image.

[0091] The tea leaf image feature sequences corresponding to each target tea garden image in the tea garden image sequence are summarized to obtain a tea leaf image feature sequence set.

[0092] The target tea garden image refers to a tea garden image obtained sequentially from a sequence of tea garden images.

[0093] The ResNet18 network is a lightweight deep residual network structure, which is quite common and will not be described in detail here.

[0094] The multi-resolution feature extraction operation refers to the process of extracting features from the target tea garden image based on different resolutions. The initial multi-layer convolutional feature map refers to the multi-resolution feature extraction result of the target tea garden image.

[0095] The residual connection operation refers to adding skip connections between the input and output of a network layer, so that the output features are represented as the sum of the input features and the residual features learned by the network layer. Residual connections are a functional structure of the ResNet18 network.

[0096] The multi-layer feature map refers to the residual connection result of the initial multi-layer convolutional feature map.

[0097] The GD-Tea feature fusion network is a multi-scale feature fusion strategy for tea bud images, which integrates low-level spatial features and high-level semantic features through a dynamic guidance mechanism.

[0098] The low-high semantic information fusion refers to the process of generating feature maps of different levels (such as C3, C4, C5) through the backbone (such as ResNet) to cover scales from low to high, then upsampling the high-level semantic features, calculating the attention correlation with the low-level features, and highlighting the key features of the bud region.

[0099] The semantic fusion multi-layer feature map refers to the result of fusing low- and high semantic information from the multi-layer feature map.

[0100] The feature concatenation refers to the process of combining multiple feature data together to form a single feature. The multi-scale feature map refers to the feature concatenation result of semantic fusion multi-layer feature maps.

[0101] The RDE-Head network refers to the segmentation head. The multi-channel segmentation result refers to the pixel-level classification operation achieved by mapping the feature map to a segmentation mask with the same resolution as the input image through convolution or deconvolution operations. The tea image feature sequence refers to the multi-channel segmentation result of multi-scale feature maps.

[0102] Specifically, in this embodiment of the invention, each image needs to be processed sequentially. Therefore, only one tea garden image is acquired first to obtain the target tea garden image. Then, the ResNet18 network is used to perform multi-resolution feature extraction on the target tea garden image to obtain an initial multi-layer convolutional feature map. For example, the initial multi-layer convolutional feature map is the initial convolutional feature map corresponding to five resolution levels: A1, A2, A3, A4, and A5. The resolution levels from A1 to A5 gradually decrease, but the convolution depth increases.

[0103] Then, residual connection operations are performed on the initial multi-layer convolutional feature maps to obtain multi-layer feature maps, such as [B2, B3, B4, and B5].

[0104] Specifically, in this embodiment of the invention, [B2, B3, B4 and B5] are simultaneously imported into Low-GD in the GD-Tea feature fusion network. Low-GD fuses the low-order detail features of B2 to B5 to obtain a semantic fusion multi-layer feature map. Then, the semantic fusion multi-layer feature map is re-fused into B3 to B5 to obtain a multi-scale feature map.

[0105] Then, using the segmentation head of the RDE-Head network, multi-channel segmentation is performed on the multi-scale feature map according to the classification types of channels such as bud, first leaf, internode below the first leaf, and second leaf, to obtain the tea image feature sequence of the target tea garden image. Then, the tea image feature sequences corresponding to each target tea garden image are summarized to obtain the tea image feature sequence set.

[0106] In detail, in this embodiment of the invention, the step of using a pre-constructed GD-Tea feature fusion network to perform low-to-high semantic information fusion on the multi-layer feature map to obtain a semantically fused multi-layer feature map, and then performing feature concatenation on the semantically fused multi-layer feature map to obtain a multi-scale feature map, includes:

[0107] Using the pre-built cascaded group attention module in the pre-built GD-Tea feature fusion network, the target deep feature map in the multi-layer feature map is subjected to cascaded group attention enhancement operation to obtain the target deep enhanced feature map;

[0108] Using the low-order guidance module pre-built in the GD-Tea feature fusion network, target shallow feature extraction is performed on the multi-layer feature map to obtain shallow features;

[0109] Using a pre-built feature enhancement module, feature enhancement operations are performed on the shallow feature fusion multi-layer feature map to obtain a shallow feature enhanced multi-layer feature map;

[0110] Using a pre-built feature enhancement module, feature enhancement operations are performed on the shallow feature enhancement multi-layer feature map to obtain a shallow feature enhancement multi-layer feature map;

[0111] Using the high-order guidance module pre-built in the GD-Tea feature fusion network, target deep feature extraction operation is performed on the shallow feature enhancement multi-layer feature map and the target deep enhancement feature map to obtain deep features;

[0112] Using the deep features, cross-scale connections are made between the second target layer feature map in the shallow feature enhancement multi-layer feature map to obtain a deep feature fusion feature map, and cross-scale connections are made between the target deep enhancement feature map to obtain a target deep fusion feature map.

[0113] Using the feature enhancement module, feature enhancement operations are performed on the deep feature fusion feature map to obtain a deep feature enhanced feature map, and feature enhancement operations are performed on the target deep fusion feature map to obtain a target deep fusion enhanced feature map;

[0114] The feature maps of the deep feature enhancement map, the target deep fusion enhancement map, and the third target layer feature map in the shallow feature enhancement multi-layer feature map are spliced ​​together to obtain a multi-scale feature map.

[0115] The cascaded grouped attention module refers to Cascaded Grouped Fusion Attention (CGFA), which is an improvement on the traditional multi-head self-attention (MHSA). Through slice localization and cascading of prior information, it enhances feature interaction and reduces computational redundancy, making it more suitable for scenarios where tea leaves are small, dense, and difficult to distinguish in this scheme.

[0116] The target deep feature map refers to the B5 feature map mentioned above.

[0117] The cascaded attention enhancement operation refers to the process of CGFA processing B5. The target deep enhancement feature map refers to the result of the CGFA operation on B5, which can be represented by P5 labels.

[0118] The low-level guidance module refers to Low-Level Guidance (Low-GD), which is used to leverage the high resolution advantage of shallow features to guide deep features to recover detailed information (such as leaf edges and small targets), thus alleviating the "edge blurring" problem in segmentation.

[0119] The target shallow feature extraction operation refers to the processing of multi-layer feature maps by Low-GD. The shallow features refer to the processing results of the multi-layer feature maps by Low-GD.

[0120] The first target layer features refer to B3 and B4 mentioned above.

[0121] The cross-scale connection refers to the Inject operation, which is used to inject guiding information from shallow features into B3 and B4. The shallow feature fusion multi-layer feature map refers to the result of B3 and B4 after shallow feature injection.

[0122] The feature enhancement module refers to the Dilated Residual Block with Concatenation, which can expand the receptive field without reducing the resolution through multi-scale dilated convolution, thereby enhancing the feature capture capability of targets at different scales such as plant leaves and branches.

[0123] The feature enhancement operation refers to the processing of shallow feature fusion multi-layer feature maps by the feature enhancement module. The shallow feature enhancement multi-layer feature map refers to the feature enhancement result of the shallow feature fusion multi-layer feature map, which can be represented by P3 and P4 labels.

[0124] The high-level guidance module refers to High-GD (High-Level Guidance), which is used to guide the learning of mid-level features (P3~P5) through high-level semantic features, thereby enhancing the understanding of the overall structure (such as plant morphology).

[0125] The target deep feature extraction operation refers to the process by which High-GD processes P3 to P5. The deep features refer to the results of High-GD processing P3 to P5.

[0126] The second target layer feature map refers to P4.

[0127] The deep feature fusion feature map refers to the result of adding deep features to P4. The target deep fusion feature map refers to the result of adding deep features to P5.

[0128] The deep feature enhancement map refers to the result of the deep feature fusion map after passing through the feature enhancement module, and can be represented by N5. The target deep fusion enhancement map refers to the result of the target deep fusion feature map after passing through the feature enhancement module, and can be represented by N4.

[0129] The third target layer feature map refers to P3.

[0130] Specifically, in this embodiment of the invention, firstly, a cascaded attention enhancement operation is performed on B5 through a cascaded attention module to obtain a target deep enhanced feature map P5. Then, a target shallow feature extraction operation is performed on the multi-layer feature map using a low-order guidance module to obtain shallow features. The shallow features are then connected across scales to B3 and B4 to obtain a shallow feature fusion multi-layer feature map.

[0131] Then, the shallow feature fusion multi-layer feature map is enhanced using a cascaded attention module to obtain shallow feature enhanced multi-layer feature maps P3 and P4. Once P3~P5 are obtained, they are uniformly fed into a higher-order guidance module to obtain deep features. Then, through cross-scale connections, the deep features are imported into P4 and P5 respectively to form a deep feature fusion feature map and a target deep fusion feature map. Then, feature enhancement operations are performed on the deep feature fusion feature map and the target deep fusion feature map respectively to obtain a deep feature enhanced feature map N5 and a target deep fusion enhanced feature map N4. In this embodiment of the invention, P3 is directly used as N3, and N3~N5 are concatenated to obtain a multi-scale feature map.

[0132] In detail, in this embodiment of the invention, the step of using the feature enhancement module to perform feature enhancement operations on the deep feature fusion feature map to obtain a deep feature enhanced feature map includes:

[0133] Using the feature enhancement module, a dilated convolution operation based on a preset number of branches is performed on the deep feature fusion feature map according to a preset set of dilated convolution parameters to obtain a dilated convolution feature map sequence.

[0134] Batch normalization is performed on each dilated convolutional feature map in the dilated convolutional feature map sequence to obtain a normalized dilated convolutional feature map sequence.

[0135] A feature fusion operation is performed on each standardized dilated convolutional feature map in the standardized dilated convolutional feature map sequence to obtain a deep feature enhancement feature map.

[0136] The set of dilated convolution parameters refers to the parameters that control the convolution of each hole.

[0137] The preset quantity refers to 3.

[0138] The dilated convolution operation based on a preset number of parallel branches refers to the process of performing dilated convolution operations on three branches simultaneously on the deep feature fusion feature map. The dilated convolution feature map sequence refers to the result of the dilated convolution operation on the deep feature fusion feature map.

[0139] The batch normalization operation refers to the process of eliminating fluctuations in the distribution of input data through normalization, which can be achieved through Batch Normalization (BN). It should be understood that plant image segmentation scenarios (such as pixel-level segmentation of tea shoots, leaves, and stems) face challenges such as large variations in lighting, diverse target morphologies, and complex backgrounds. BN improves network performance by mitigating the impact of lighting and color fluctuations, enhancing the extraction of small targets and detailed features, and improving the network's robustness to complex backgrounds.

[0140] The standardized dilated convolutional feature map sequence refers to the result of each dilated convolutional feature map being processed by BN.

[0141] The feature fusion operation refers to the Add (+) operation, which adds the pixel values ​​at corresponding positions of two or more feature maps with identical shapes, resulting in an output feature map whose shape matches the input feature map. Figure 1 The deep feature enhancement feature map refers to the result of feature fusion operations performed on various standardized dilated convolutional feature maps.

[0142] In detail, in this embodiment of the invention, the step of performing a pickable tender shoot identification operation on the tea image feature sequence set to obtain a tender shoot object sequence set includes:

[0143] The tea leaf image feature sequence set is subjected to tender shoot contour recognition to obtain a set of tender shoot bounding boxes;

[0144] Based on the pre-constructed Kalman filter algorithm, the motion state of each tender shoot outer frame in the tender shoot outer frame set is monitored to obtain a tracker set;

[0145] Based on the pre-constructed Hungarian matching association algorithm, the set of trackers is used to perform a deduplication operation on the set of tender shoot bounding boxes to obtain a set of tender shoot object sequences.

[0146] The aforementioned tender shoot contour recognition refers to the process of segmenting tender shoots using an edge segmentation algorithm and marking them with a minimum bounding box.

[0147] The set of outer bounding boxes for tender shoots refers to the set of marked boxes for each identified tender shoot in the tea leaf image feature sequence set.

[0148] The Kalman filter algorithm is a recursive estimation algorithm based on the state equation of a linear system and the Gaussian noise assumption. It is suitable for state tracking and prediction of real-time dynamic systems (such as target tracking and navigation positioning of each shoot).

[0149] The motion state refers to the state in which the shape of the same tender shoot changes in different tea leaf images when the drone's perspective changes.

[0150] The monitoring refers to the process of maintaining a Kalman filter (tracker) for each tracked shoot to predict the current frame position and update the actual state. The tracker set refers to the set of Kalman filters that track each shoot.

[0151] The Hungarian matching association algorithm is a combinatorial optimization algorithm for finding the optimal matching of a bipartite graph.

[0152] The deduplication operation refers to: determining whether each tender shoot outer frame in the tender shoot outer frame set has been tracked according to the Hungarian matching association algorithm; if the tender shoot outer frame has not been tracked, then the untracked tender shoot outer frame is tracked, thereby updating the tracker set.

[0153] The set of tender shoot object sequences refers to the set of tender shoot outer frames after the deduplication operation.

[0154] Specifically, in this embodiment of the invention, the tea image feature sequence set is subjected to tender shoot contour recognition to obtain a tender shoot bounding box set. Each tender shoot bounding box in the tender shoot bounding box set may be a tender shoot from a tea garden image in different frames. Since there are changes in shooting position and angle between different tea garden images, a tender shoot may appear in different tea garden images. Therefore, the tender shoot bounding box set has a high degree of repeatability.

[0155] In this embodiment of the invention, the outer bounding boxes of tender shoots in the first frame of a tea garden image are first obtained in time frame order. A Kalman filter algorithm is then used to monitor all the outer bounding boxes of tender shoots in the first frame of the tea garden image, resulting in a first tracker set. Next, the outer bounding boxes of tender shoots in the second frame of the tea garden image are obtained. A Hungarian matching association algorithm is used to determine whether the outer bounding boxes of tender shoots in the second frame of the tea garden image already exist in the tracker set. Each outer bounding box of tender shoots in the second frame of the tea garden image is divided into tracked and untracked, thereby updating the first tracker set. This process is repeated to remove duplicates from the set of outer bounding boxes of tender shoots, resulting in a sequence set of tender shoot objects.

[0156] S2. Using a pre-constructed deep learning feature extraction network, extract the tea image features corresponding to each tender shoot object in the tender shoot object sequence set from the tea garden image sequence to obtain the tender shoot fingerprint feature sequence set.

[0157] The deep learning feature extraction network mentioned here refers to SuperPoint, which replaces SIFT in Colmap. Colmap is software in the field of 3D reconstruction that transforms 2D images into 3D structures through visual feature association and geometric optimization.

[0158] The set of tender shoot fingerprint feature sequences refers to the tea image features corresponding to each tender shoot object.

[0159] Specifically, in this embodiment of the invention, FeatureExtractor.use_cuda 1 is enabled in Colmap (requires support for deep learning modules at compile time), and SuperPoint.nms_radius 4 is set (to suppress neighboring redundant feature points) and SuperPoint.confidence_threshold 0.01 (to reduce the confidence threshold to retain more weak feature points).

[0160] The deep learning feature extraction network of this invention has higher repeatability (stable detection of the same tender shoot under different viewpoints) and discriminability (distinguishing between leaf texture and bud tip details), and is especially suitable for scenes with dense small targets.

[0161] Specifically, in this embodiment of the invention, the process in S1 is used to distinguish tender buds. Next, it is necessary to identify the characteristics of the tender buds. Therefore, this invention extracts the tea image features corresponding to each tender bud object in the tender bud object sequence set from the tea garden image sequence, obtaining a tender bud fingerprint feature sequence set. This tender bud fingerprint feature sequence set can provide data support for subsequent identification of tender bud weight.

[0162] S3. Using a pre-constructed deep feature matching network, based on the set of tender shoot fingerprint feature sequences, the corresponding relationship of tea tree tender shoots between tea garden images in the tea garden image sequence is matched to obtain the tea tree tender shoot matching relationship.

[0163] The deep feature matching network mentioned here refers to SuperGlue. This invention utilizes SuperGlue to replace FLANN in Colmap to achieve end-to-end geometry-aware matching.

[0164] The corresponding relationship matching is the process of identifying the relationship between the tender shoots of each tea tree in each tea garden image.

[0165] The tea tree shoot matching relationship refers to the set of relationships between tea tree shoots in each tea garden image.

[0166] Specifically, in this embodiment of the invention, Matcher.matcher_type super_glue is enabled in Colmap, and SuperGlue.match_threshold is set to 0.8 (to increase the matching confidence threshold). The geometric relationship between feature points is learned using the Transformer architecture, and "matching pairs with confidence" are directly output, reducing mismatches (especially suitable for repetitive texture scenes).

[0167] Specifically, in this embodiment of the invention, a deep feature matching network is executed based on the set of tender shoot fingerprint features to match the corresponding relationships of tea tree tender shoots between various tea garden images in the tea garden image sequence, thereby obtaining the matching relationship of tea tree tender shoots.

[0168] S4. Using pre-built camera pose estimation software, calculate the viewpoint-position relationship of each tea garden image in the tea garden image sequence according to the matching relationship of the tea tree shoots, obtain the viewpoint-position relationship set, and perform modeling and reconstruction operation on the shoot object sequence set according to the viewpoint-position relationship set to obtain the shoot object model set, and obtain the key information sequence of each shoot object model in the shoot object model set to obtain the key information sequence set.

[0169] The camera pose estimation software mentioned above refers to Colmap.

[0170] The process of calculating the viewpoint-position relationship of each tea garden image in the tea garden image sequence based on the matching relationship of the tea tree shoots, and obtaining the viewpoint-position relationship set, is an automated process of Colmap, which will not be elaborated here.

[0171] The viewpoint-position relationship refers to the combination of the shooting viewpoint of the tea garden image and the positional information of each tender shoot in the tea garden image.

[0172] The modeling and reconstruction operation refers to the process of transforming the tender shoot objects of each tea garden image from a two-dimensional image into a three-dimensional image based on the viewpoint-position relationship of each tea garden image.

[0173] The set of tender shoot object models refers to the set of tender shoot objects that have been successfully modeled.

[0174] The key information sequence refers to the bud length, the area of ​​the first leaf, the length of the internode below the first leaf, the area of ​​the second leaf, and the length of the internode below the second leaf. The key information sequence set refers to the set that stores the key information sequences.

[0175] Specifically, in this embodiment of the invention, the camera pose is optimized directly using the bundle adjustment (BA) method built into the Colmap tool to obtain the viewpoint-position relationship set.

[0176] In detail, in this embodiment of the invention, the step of performing a modeling and reconstruction operation on the set of tender shoot object sequences based on the viewpoint-position relationship set to obtain a set of tender shoot object models includes:

[0177] Using the pre-built position encoder in the camera pose estimation software, the viewpoint-position relationship set is encoded based on mapping to three-dimensional space to obtain a three-dimensional position encoding vector;

[0178] Using a pre-constructed two-layer multilayer perceptron, a radiation field attribute prediction operation is performed on the three-dimensional position encoding vector to obtain a three-dimensional representation set of tender shoots;

[0179] Based on the pre-constructed CtF sampling strategy, the three-dimensional representation set of the tender shoots is modeled to obtain a set of tender shoot object models.

[0180] The position encoder refers to Position Encoding, a technology used to inject position information into sequence data.

[0181] The encoding operation based on mapping to three-dimensional space refers to the process of transforming the position coordinates of each shoot and the shooting viewpoint coordinates from two dimensions in the image into three dimensions in the model. The three-dimensional position encoding vector refers to the set of viewpoint-position relationships obtained through the position encoder.

[0182] The dual-layer multilayer perceptron refers to two multilayer perceptrons (MLP network structure).

[0183] The radiation field attribute prediction operation refers to the following: the first layer MLP processes the coordinate information and outputs the volume density σ and intermediate features of the model, and the second layer MLP combines the observation direction to output the color c of the model, thereby obtaining the three-dimensional representation set of the tender shoot.

[0184] The three-dimensional representation set of the tender shoots refers to the prediction result of the three-dimensional position encoding vector by the two-layer multilayer perceptron.

[0185] The CtF sampling strategy refers to the Coarse-to-Fine sampling strategy, which generates images through ray casting, i.e., emitting light rays from the camera center, sampling 3D points along the light rays, and integrating the color. To avoid sampling redundancy, a two-step strategy of "coarse sampling + fine sampling" is adopted: Coarse sampling: 128 points are uniformly sampled along the light ray. The MLP predicts the volume density σ and color c, and the "coarse color" and "importance weight" of the light ray are calculated through volume rendering (regions with high σ have high weight). Fine sampling: Based on the importance weight of the coarse sampling, 128 sampling points are added to high-weight regions (such as object surfaces), and the fine color is recalculated.

[0186] The rendering refers to the method of generating a visual image of the tender shoot object model.

[0187] Specifically, this invention first utilizes a position encoder to transform a two-dimensional viewpoint-position relationship set into a three-dimensional position encoding vector. Then, a two-layer, multi-layer perception layer is used to perform radiation field attribute prediction on the three-dimensional position encoding vector to obtain a three-dimensional representation set of the shoot. In this embodiment of the invention, the three-dimensional representation set of the shoot can already represent the three-dimensional structure of the shoot, and the model can be directly rendered. However, to improve rendering efficiency, this invention employs a CtF sampling strategy for the rendering process.

[0188] Specifically, in this embodiment of the invention, after obtaining the set of tender shoot object models, the key information sequence of each model in the set of tender shoot object models can be directly obtained to obtain a set of key information sequences.

[0189] S5. Based on the pre-constructed yield conversion experience model and the set of key information sequences, calculate the weight composition sequence of each tender shoot object model in the set of tender shoot object models to obtain the set of weight composition sequences.

[0190] The aforementioned production conversion empirical model refers to a model that can be used for production conversion, obtained through regression training based on actual data.

[0191] The weight composition sequence refers to the bud weight (dry and wet weight, including the weight of dried leaves and the weight of fresh leaves), the weight of the first leaf (dry and wet weight), the weight of the second leaf (dry and wet weight), the weight of the stem below the first leaf (dry and wet weight), and the weight of the stem below the second leaf (dry and wet weight). The set of weight composition sequences refers to the collection of all weight composition sequences.

[0192] In detail, in this embodiment of the invention, before the step of using the pre-constructed production conversion empirical model and the set of key information sequences, the method further includes:

[0193] Obtain tea leaf sample data;

[0194] The target parameter relationship regression fitting operation was performed on the tea leaf sample data to obtain the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight.

[0195] Models were constructed for the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight to obtain an empirical model for yield conversion.

[0196] The tea leaf sample data refers to the actual measurement results of a large amount of field data.

[0197] The target parameter relationship regression fitting operation refers to the process of fitting the relationship between key information sequences.

[0198] The relationships described are as follows: The bud fresh weight-bud length relationship refers to the correspondence between bud length and bud fresh weight. The bud fresh weight-bud dry weight relationship refers to the change in bud fresh weight after drying. The first leaf area-first leaf fresh weight relationship refers to the relationship between the first leaf fresh weight and the first leaf area. The first leaf fresh weight-first leaf dry weight relationship refers to the change in first leaf fresh weight and first leaf dry weight. The first leaf lower internode length-first leaf lower internode fresh weight relationship refers to the relationship between the first leaf lower internode length and the first leaf lower internode fresh weight. The first leaf lower internode fresh weight-first leaf lower internode dry weight relationship refers to the relationship between the first leaf lower internode fresh weight and the first leaf lower internode dry weight. The second leaf area-second leaf fresh weight relationship refers to the relationship between the second leaf area and the second leaf fresh weight. The second leaf dry weight-second leaf fresh weight relationship refers to the relationship between the second leaf dry weight and the second leaf fresh weight. The second leaf lower internode length-second leaf lower internode fresh weight relationship refers to the relationship between the second leaf lower internode length and the second leaf lower internode fresh weight. The relationship between the fresh weight and dry weight of the internode below the second leaf refers to the relationship between the fresh weight and dry weight of the internode below the second leaf.

[0199] The model construction refers to the process of using the above relationship as the output layer function of the model, which can transform observable data (length, area) into weight.

[0200] Specifically, in this embodiment of the invention, the key information sequence set can be transformed from observation-type data into weights through the above relationship, thereby obtaining a weight-based sequence set.

[0201] S6. Based on the pre-constructed seasonal influence parameters, perform a tea yield estimation operation on the set of weight composition sequences to obtain the estimated tea yield.

[0202] The seasonal influence parameter refers to the parameter that affects the above relationships according to changes over time.

[0203] The tea yield estimation operation refers to the process of calculating the weight of the required portion of each tender shoot.

[0204] The estimated yield of tea refers to the result of calculating the weight of the required portion of each tender shoot.

[0205] In detail, in this embodiment of the invention, before proceeding with the step of using pre-constructed seasonal influence parameters, the method further includes:

[0206] Acquire tea leaf data samples from different seasons, configure initial seasonal parameters for the yield conversion empirical model, and obtain the seasonally initialized yield conversion empirical model.

[0207] Using the aforementioned seasonal initialization yield conversion empirical model, predictions are made on tea leaf data samples from different seasons to obtain a predicted tea leaf weight sequence.

[0208] Calculate the loss value between the pre-constructed real tea leaf weight sequence and the predicted tea leaf weight sequence in the tea leaf data samples of different seasons;

[0209] Minimize the loss value to obtain the network model parameters with the minimum loss value, and use the network model parameters to update the initial seasonal parameters to obtain the seasonal influence parameters.

[0210] The tea leaf data samples from different seasons refer to tea leaf sample data obtained at different times and seasons.

[0211] The initial seasonal parameters are unknowns configured in the aforementioned relationships.

[0212] The seasonal initialization yield conversion empirical model refers to a conversion empirical model configured with initial seasonal parameters.

[0213] The prediction refers to the forward execution process of the seasonally initialized yield conversion empirical model.

[0214] The predicted tea leaf weight sequence refers to the prediction results of the seasonally initialized yield conversion empirical model for tea leaf data samples from different seasons.

[0215] The loss value refers to the calculation result of the pre-constructed real tea leaf weight sequence and the predicted tea leaf weight sequence in the tea leaf data samples of different seasons under the cross-entropy loss algorithm.

[0216] The process of minimizing the loss value to obtain the network model parameters with the minimum loss value is a common structure in the neural network training process, and will not be elaborated here.

[0217] The seasonal influence parameter refers to the initial seasonal parameter that is updated by the network model parameters.

[0218] Specifically, in this embodiment of the invention, Yinghong No. 9 is mainly cultivated in the south, with a harvesting cycle that can last from March to November, and eight harvests can be made each year. Considering that harvesting in different seasons has a significant impact on the estimation of tea leaf weight, this invention uses tea leaf data samples from different seasons to fine-tune and train the yield conversion empirical model to obtain seasonal influence parameters.

[0219] Specifically, in this embodiment of the invention, a training method using cross-entropy loss algorithm and gradient descent algorithm is adopted to calculate the loss value and minimize the loss value to obtain the network model parameters when the loss value is minimized. Then, through reverse network update, a trained production conversion empirical model is obtained, as well as seasonal influence parameters.

[0220] To address the problems described in the background art, this invention first identifies the tender shoot objects and their quantity in tea garden images through object detection. Then, it uses Colmap to perform refined modeling of the tender shoot objects, thereby estimating the weight of the tender shoots. However, to adapt to situations where tea leaves are small, numerous, and the characteristics of the tender shoots are similar to those of surrounding tea leaves, this invention improves the modeling process by employing an RT-DETR detection model based on the Transformer framework. It replaces the multi-head self-attention (MHSA) mechanism in the AIFI module with cascaded grouped attention (CGFA). Through slicing and cascading operations, each attention head acquires local information... Simultaneously, all information from the preceding header is obtained, deep features are optimized, semantic information is enriched, and computational redundancy is reduced. Furthermore, the GD-Tea feature fusion mechanism effectively integrates low-level and high-level semantic information from tea bud images, ensuring the model can accurately identify tea buds in complex environments. This invention also combines the Hungarian matching algorithm and the Kalman filter tracking algorithm to accurately count the harvestable tea shoots in tea garden image sequences from video. Additionally, this invention adaptively improves Colmap by replacing the traditional SIFT with a deep learning feature extraction network and the ANN matching module with a deep feature matching network, thereby improving the accuracy of shoot modeling. Therefore, this invention can improve the accuracy and efficiency of tea yield estimation.

[0221] like Figure 2 The diagram shown is a functional block diagram of a digital in-situ yield estimation device for Yinghong No. 9 tea gardens provided in an embodiment of the present invention.

[0222] The digital in-situ yield estimation device 100 for Yinghong No. 9 tea gardens described in this invention can be installed in an electronic device. Depending on the functions implemented, the digital in-situ yield estimation device 100 for Yinghong No. 9 tea gardens may include a target detection module 101, a tea leaf modeling module 102, a weight calculation module 103, and a seasonal adjustment module 104. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and which are stored in the memory of the electronic device.

[0223] The target detection module 101 is used to acquire a tea garden image sequence and identify the pluckable tender shoots in each tea garden image in the tea garden image sequence to obtain a set of tender shoot object sequences;

[0224] The tea leaf modeling module 102 is used to extract tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set from the tea garden image sequence using a pre-constructed deep learning feature extraction network, to obtain a tender shoot fingerprint feature sequence set; and to perform correspondence matching of tea tree tender shoots between tea garden images in the tea garden image sequence according to the tender shoot fingerprint feature sequence set using a pre-constructed deep feature matching network, to obtain tea tree tender shoot matching relationships; and to calculate the viewpoint-position relationship of each tea garden image in the tea garden image sequence according to the tea tree tender shoot matching relationships using pre-constructed camera pose estimation software, to obtain a viewpoint-position relationship set; and to perform modeling and reconstruction operations on the tender shoot object sequence set according to the viewpoint-position relationship set, to obtain a tender shoot object model set; and to obtain the key information sequence of each tender shoot object model in the tender shoot object model set, to obtain a key information sequence set.

[0225] The weight calculation module 103 is used to calculate the weight composition sequence of each tender shoot object model in the tender shoot object model set based on the pre-constructed yield conversion experience model and the key information sequence set, so as to obtain the weight composition sequence set.

[0226] The seasonal adjustment module 104 is used to perform tea yield estimation operation on the set of weight composition sequences based on pre-constructed seasonal influence parameters to obtain the estimated tea yield.

[0227] In detail, the modules in the digital in-situ yield estimation device 100 for Yinghong No. 9 tea gardens described in this embodiment of the invention employ the same methods as described above during use. Figure 1 The method described herein uses the same technical means as the digital in-situ yield estimation method for Yinghong No. 9 tea gardens and can produce the same technical effects, so it will not be elaborated here.

[0228] like Figure 3The diagram shown is a schematic representation of the structure of an electronic device for implementing a digital in-situ yield estimation method for Yinghong No. 9 tea gardens, according to an embodiment of the present invention.

[0229] The electronic device 1 may include a processor 10, a memory 11 and a bus 12, and may also include a computer program stored in the memory 11 and capable of running on the processor 10, such as the digital in-situ yield estimation method program for Yinghong No. 9 tea garden.

[0230] The memory 11 includes at least one type of readable storage medium, such as flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as the portable hard drive of the electronic device 1. In other embodiments, the memory 11 can be an external storage device of the electronic device 1, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 1. Furthermore, the memory 11 includes both internal storage units and external storage devices of the electronic device 1. The memory 11 can be used not only to store application software and various types of data installed on the electronic device 1, such as the code of the digital in-situ yield estimation method program for Yinghong No. 9 tea garden, but also to temporarily store data that has been output or will be output.

[0231] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (such as the digital in-situ yield estimation method program for Yinghong No. 9 tea gardens) and calls data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

[0232] The bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus 12 can be divided into an address bus, a data bus, a control bus, etc. The bus 12 is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0233] Figure 3 Only electronic devices with components are shown; those skilled in the art will understand that... Figure 3 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0234] For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0235] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, a Bluetooth interface, etc.), which is typically used to establish communication connections between the electronic device 1 and other electronic devices.

[0236] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), and optionally, a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device 1 and to display a visual user interface.

[0237] The digital in-situ yield estimation method program for Yinghong No. 9 tea gardens stored in the memory 11 of the electronic device 1 is a combination of multiple instructions. When run in the processor 10, it can achieve the following:

[0238] Acquire a sequence of tea garden images and identify the pluckable tender shoots in each tea garden image in the sequence to obtain a set of tender shoot object sequences and a set of tender shoot quantities;

[0239] Using a pre-built deep learning feature extraction network, tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set are extracted from the tea garden image sequence to obtain a tender shoot fingerprint feature sequence set.

[0240] Using a pre-constructed deep feature matching network, the corresponding relationship of tea tree shoots between tea garden images in the tea garden image sequence is matched according to the set of tender shoot fingerprint feature sequences, so as to obtain the tea tree shoot matching relationship;

[0241] Using pre-built camera pose estimation software, the viewpoint-position relationship of each tea garden image in the tea garden image sequence is calculated based on the matching relationship of the tea tree shoots, resulting in a viewpoint-position relationship set. Based on the viewpoint-position relationship set, the shoot object sequence set is modeled and reconstructed to obtain a shoot object model set. The key information sequence of each shoot object model in the shoot object model set is then obtained to obtain a key information sequence set.

[0242] Based on the pre-constructed yield conversion experience model and the set of key information sequences, the weight composition sequence of each tender shoot object model in the set of tender shoot object models is calculated to obtain the set of weight composition sequences.

[0243] Based on the pre-constructed seasonal influence parameters and the set of tender shoot quantities, the tea yield is estimated by performing a tea yield estimation operation on the set of weight composition sequences.

[0244] Specifically, the processor 10's implementation method for the above instructions can be found in [reference needed]. Figures 1 to 3 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0245] Furthermore, if the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0246] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following:

[0247] Acquire a sequence of tea garden images and identify the pluckable tender shoots in each tea garden image in the sequence to obtain a set of tender shoot object sequences and a set of tender shoot quantities;

[0248] Using a pre-built deep learning feature extraction network, tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set are extracted from the tea garden image sequence to obtain a tender shoot fingerprint feature sequence set.

[0249] Using a pre-constructed deep feature matching network, the corresponding relationship of tea tree shoots between tea garden images in the tea garden image sequence is matched according to the set of tender shoot fingerprint feature sequences, so as to obtain the tea tree shoot matching relationship;

[0250] Using pre-built camera pose estimation software, the viewpoint-position relationship of each tea garden image in the tea garden image sequence is calculated based on the matching relationship of the tea tree shoots, resulting in a viewpoint-position relationship set. Based on the viewpoint-position relationship set, the shoot object sequence set is modeled and reconstructed to obtain a shoot object model set. The key information sequence of each shoot object model in the shoot object model set is then obtained to obtain a key information sequence set.

[0251] Based on the pre-constructed yield conversion experience model and the set of key information sequences, the weight composition sequence of each tender shoot object model in the set of tender shoot object models is calculated to obtain the set of weight composition sequences.

[0252] Based on the pre-constructed seasonal influence parameters and the set of tender shoot quantities, the tea yield is estimated by performing a tea yield estimation operation on the set of weight composition sequences.

[0253] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, and actual implementations may have other classification methods.

[0254] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0255] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0256] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0257] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A digital in-situ yield estimation method for Yinghong No. 9 tea gardens, characterized in that, The method includes: A sequence of tea garden images is obtained, and the pluckable tender shoots in each tea garden image in the sequence are identified to obtain a set of tender shoot object sequences. Using a pre-built deep learning feature extraction network, tea image features corresponding to each tender shoot object in the tender shoot object sequence set are extracted from the tea garden image sequence to obtain a tender shoot fingerprint feature sequence set. Using a pre-constructed deep feature matching network, the corresponding relationship of tea tree shoots between tea garden images in the tea garden image sequence is matched according to the set of tender shoot fingerprint feature sequences, so as to obtain the tea tree shoot matching relationship; Using pre-built camera pose estimation software, the viewpoint-position relationship of each tea garden image in the tea garden image sequence is calculated based on the matching relationship of the tea tree shoots, resulting in a viewpoint-position relationship set. Based on the viewpoint-position relationship set, the shoot object sequence set is modeled and reconstructed to obtain a shoot object model set. The key information sequence of each shoot object model in the shoot object model set is then obtained to obtain a key information sequence set. Based on the pre-constructed yield conversion experience model and the set of key information sequences, the weight composition sequence of each tender shoot object model in the set of tender shoot object models is calculated to obtain the set of weight composition sequences. Based on the pre-constructed seasonal influence parameters, the tea yield is estimated by performing a tea yield estimation operation on the set of weight composition sequences. The empirical model for output conversion is constructed using the following method: Obtain tea leaf sample data; The target parameter relationship regression fitting operation was performed on the tea leaf sample data to obtain the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight. Models were constructed for the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight to obtain an empirical model for yield conversion.

2. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 1, characterized in that, The process of acquiring a sequence of tea garden images and identifying harvestable tender shoots in each tea garden image within the sequence to obtain a set of tender shoot object sequences includes: Acquire videos of tender tea shoots per unit area of ​​tea garden during the harvest season, and perform frame extraction processing on the videos of tender tea shoots to obtain a tea garden image sequence; Using a pre-constructed target detection network, feature extraction is performed on each tea garden image in the tea garden image sequence to obtain a tea leaf image feature sequence set. The tea image feature sequence set is subjected to a pickable tender shoot identification operation to obtain a tender shoot object sequence set.

3. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 2, characterized in that, The method utilizes a pre-constructed target detection network to perform feature extraction operations on each tea garden image in the tea garden image sequence, obtaining a tea leaf image feature sequence set, including: The target tea garden image is obtained by sequentially acquiring one tea garden image from the tea garden image sequence. The target tea garden image is subjected to multi-resolution feature extraction using the pre-built ResNet18 network in the pre-built target detection network to obtain an initial multi-layer convolutional feature map, and a residual connection operation is performed on the initial multi-layer convolutional feature map to obtain a multi-layer feature map. Using the pre-constructed GD-Tea feature fusion network, low-high semantic information is fused into the multi-layer feature map to obtain a semantically fused multi-layer feature map, and the semantically fused multi-layer feature map is then concatenated to obtain a multi-scale feature map. The multi-scale feature map is segmented using a pre-constructed RDE-Head network to obtain the tea leaf image feature sequence of the target tea garden image; The tea leaf image feature sequences corresponding to each target tea garden image in the tea garden image sequence are summarized to obtain a tea leaf image feature sequence set.

4. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 3, characterized in that, The pre-constructed GD-Tea feature fusion network is used to fuse low-to-high semantic information in the multi-layer feature map to obtain a semantically fused multi-layer feature map. Then, the semantically fused multi-layer feature map is concatenated to obtain a multi-scale feature map, including: Using the pre-built cascaded group attention module in the pre-built GD-Tea feature fusion network, the target deep feature map in the multi-layer feature map is subjected to cascaded group attention enhancement operation to obtain the target deep enhanced feature map; Using the low-order guidance module pre-built in the GD-Tea feature fusion network, target shallow feature extraction is performed on the multi-layer feature map to obtain shallow features; Using the shallow features, a cross-scale connection is performed on the preset first target layer feature map in the multi-layer feature map to obtain a shallow feature fusion multi-layer feature map; Using a pre-built feature enhancement module, feature enhancement operations are performed on the shallow feature fusion multi-layer feature map to obtain a shallow feature enhanced multi-layer feature map; Using the high-order guidance module pre-built in the GD-Tea feature fusion network, target deep feature extraction operation is performed on the shallow feature enhancement multi-layer feature map and the target deep enhancement feature map to obtain deep features; Using the deep features, cross-scale connections are made between the second target layer feature map in the shallow feature enhancement multi-layer feature map to obtain a deep feature fusion feature map, and cross-scale connections are made between the target deep enhancement feature map to obtain a target deep fusion feature map. Using the feature enhancement module, feature enhancement operations are performed on the deep feature fusion feature map to obtain a deep feature enhanced feature map, and feature enhancement operations are performed on the target deep fusion feature map to obtain a target deep fusion enhanced feature map; The feature maps of the deep feature enhancement map, the target deep fusion enhancement map, and the third target layer feature map in the shallow feature enhancement multi-layer feature map are spliced ​​together to obtain a multi-scale feature map.

5. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 4, characterized in that, The step of using the feature enhancement module to perform feature enhancement operations on the deep feature fusion feature map to obtain a deep feature enhanced feature map includes: Using the feature enhancement module, a dilated convolution operation based on a preset number of branches is performed on the deep feature fusion feature map according to a preset set of dilated convolution parameters to obtain a dilated convolution feature map sequence. Batch normalization is performed on each dilated convolutional feature map in the dilated convolutional feature map sequence to obtain a normalized dilated convolutional feature map sequence. A feature fusion operation is performed on each standardized dilated convolutional feature map in the standardized dilated convolutional feature map sequence to obtain a deep feature enhancement feature map.

6. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 5, characterized in that, The step of performing a pickable tender shoot identification operation on the tea image feature sequence set to obtain a tender shoot object sequence set includes: The tea leaf image feature sequence set is subjected to tender shoot contour recognition to obtain a set of tender shoot bounding boxes; Based on the pre-constructed Kalman filter algorithm, the motion state of each tender shoot outer frame in the tender shoot outer frame set is monitored to obtain a tracker set; Based on the pre-constructed Hungarian matching association algorithm, the set of trackers is used to perform a deduplication operation on the set of tender shoot bounding boxes to obtain a set of tender shoot object sequences.

7. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 6, characterized in that, The step of modeling and reconstructing the tender shoot object sequence set based on the viewpoint-position relationship set to obtain a tender shoot object model set includes: Using the pre-built position encoder in the camera pose estimation software, the viewpoint-position relationship set is encoded based on mapping to three-dimensional space to obtain a three-dimensional position encoding vector; Using a pre-constructed two-layer multilayer perceptron, a radiation field attribute prediction operation is performed on the three-dimensional position encoding vector to obtain a three-dimensional representation set of tender shoots; Based on the pre-constructed CtF sampling strategy, the three-dimensional representation set of the tender shoots is modeled to obtain a set of tender shoot object models.

8. The digital in-situ yield estimation method for Yinghong No. 9 tea gardens as described in claim 7, characterized in that, Before applying the pre-constructed seasonal influence parameters, the method further includes: Acquire tea leaf data samples from different seasons, configure initial seasonal parameters for the yield conversion empirical model, and obtain the seasonally initialized yield conversion empirical model. Using the aforementioned seasonal initialization yield conversion empirical model, predictions are made on tea leaf data samples from different seasons to obtain a predicted tea leaf weight sequence. Calculate the loss value between the pre-constructed real tea leaf weight sequence and the predicted tea leaf weight sequence in the tea leaf data samples of different seasons; Minimize the loss value to obtain the network model parameters with the minimum loss value, and use the network model parameters to update the initial seasonal parameters to obtain the seasonal influence parameters.

9. A digital in-situ yield estimation device for Yinghong No. 9 tea gardens, characterized in that, The device includes: The target detection module is used to acquire a sequence of tea garden images and identify the pluckable tender shoots in each tea garden image in the sequence of tea garden images to obtain a set of tender shoot object sequences; The tea leaf modeling module is used to extract tea leaf image features corresponding to each tender shoot object in the tender shoot object sequence set from the tea garden image sequence using a pre-built deep learning feature extraction network, to obtain a tender shoot fingerprint feature sequence set; and to perform correspondence matching of tea tree tender shoots between tea garden images in the tea garden image sequence based on the tender shoot fingerprint feature sequence set, to obtain tea tree tender shoot matching relationships; and to calculate the viewpoint-position relationship of each tea garden image in the tea garden image sequence based on the tea tree tender shoot matching relationship using pre-built camera pose estimation software, to obtain a viewpoint-position relationship set; and to perform modeling and reconstruction operations on the tender shoot object sequence set based on the viewpoint-position relationship set, to obtain a tender shoot object model set; and to obtain the key information sequence of each tender shoot object model in the tender shoot object model set, to obtain a key information sequence set. The weight calculation module is used to calculate the weight composition sequence of each tender shoot object model in the tender shoot object model set based on the pre-constructed yield conversion experience model and the key information sequence set, so as to obtain the weight composition sequence set. The empirical model for output conversion is constructed using the following method: Obtain tea leaf sample data; The target parameter relationship regression fitting operation was performed on the tea leaf sample data to obtain the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight. Models were constructed for the relationships of bud fresh weight-bud length, bud fresh weight-bud dry weight, first leaf area-first leaf fresh weight, first leaf fresh weight-first leaf dry weight, first leaf lower internode length-first leaf lower internode fresh weight, first leaf lower internode fresh weight-first leaf lower internode dry weight, second leaf area-second leaf fresh weight, second leaf dry weight-second leaf fresh weight, second leaf lower internode length-second leaf lower internode fresh weight, and second leaf lower internode fresh weight-second leaf lower internode dry weight to obtain an empirical model for yield conversion. The seasonal adjustment module is used to perform tea yield estimation on the set of weight composition sequences based on pre-constructed seasonal influence parameters to obtain the estimated tea yield.

Images