A method for extracting a single plant growth period of a sparse-planting fruit tree, an electronic device, and a storage medium

Abstract

This invention relates to a method, electronic device, and storage medium for extracting the growth period of sparsely planted fruit trees, belonging to the field of image classification technology. To solve the problem of accurate classification of fruit tree growth periods, this invention uses a semantic segmentation model to predict the canopy vector surface f0 of all sparsely planted fruit trees from UAV images p0. Error correction and period attribute assignment are performed on f0 to obtain growth period labels f. Texture feature calculation is performed on p0, and the ReliefF algorithm is applied to select texture features by weight sorting. The selected texture features are added to p0 to obtain the growth period image p. f and p are paired to form a label image pair. A BDSSNet classification model for fruit tree growth periods is constructed. The label image pair is input into BDSSNet for training to obtain the fruit tree growth period classification model. The newly predicted canopy vector surface of the target area is used to classify the growth period of sparsely planted fruit trees. This invention accurately identifies the growth period status of individual fruit trees in orchards.

Description

Technical Field

[0001] This invention belongs to the field of image classification technology, specifically relating to a method for extracting the growth period of a single sparsely planted fruit tree, an electronic device, and a storage medium. Background Technology

[0002] Growing period information is an important indicator of the growth and development of fruit trees and is also necessary information for standardized cultivation and management of orchards. In digitally managed orchards, using remote sensing images to collect growing period information is more standardized and regulated than manual observation methods, and it is also easier to connect to the orchard digital management system for real-time, individual tree-based period monitoring.

[0003] With the continuous expansion of planting scale and the increasing labor costs, it is particularly important to monitor and manage the growth period of fruit trees using machine vision rather than visual interpretation. This is especially true for specialty fruit crops, where agricultural insurance often uses the growth stage of a single tree at the time of disaster as the subject of claims. Currently, methods for extracting the growth period of fruit trees based on image recognition are relatively rare. These methods mainly focus on the phenological and growth period information of economic crops, including long-term index methods and deep learning recognition methods. Long-term index methods calculate vegetation indices from multiple remote sensing images of the region of interest to obtain vegetation growth curves and infer phenological periods. However, this method is often applied to large-scale crops with strong seasonal characteristics, such as wheat and rice, for extracting their growth periods. For fruit trees, tropical fruit trees have suitable growth conditions throughout the four seasons, exhibiting timeliness, variability, and complex growth processes. For some special fruit tree varieties, farmers can even selectively induce flowering or fruiting based on the density of flowers and fruits and the price of the fruit, thus altering the growth period. These human management practices disrupt the seasonal characteristics of orchards, making long-term index methods unsuitable for tropical orchards. Whether in temperate orchards with fixed growth cycles or tropical orchards with variable periods, deep learning recognition methods can quickly, accurately, and cost-effectively extract the phenotypic features of fruit trees, obtaining the growth period of individual trees within the image area using only a single remote sensing image.

[0004] Regarding the separability of growth stages in single-scene remote sensing images, this paper analyzes the phenotypic characteristics of various possible stages of fruit tree growth. ① Pre-flowering stage: Dense, compact foliage, appearing green; ② Flowering stage: Generally, abundant flowers, compact blooms, and distinct flower colors, clearly distinguishable from other stages; ③ Pruning stage: Bare branches and trunks; fruit trees generally require post-harvest pruning and pruning during the growth period (flowering and fruiting stages), clearly distinguishable from other stages; ④ Small fruit stage: Immature fruits, due to their small weight, appear radiating outwards and fuzzy, clearly distinguishable from other stages; ⑤ Enlargement stage: The fruits are heavier, the branches are bent, exposing the central trunk. However, when identifying from above the fruit tree canopy, distinguishing between the pre-flowering stage and part of the enlargement stage is quite difficult, and most fruit trees may encompass some or all of these stages. Currently, there are few studies applying deep learning to flower identification during the growth stages of fruit trees. Summary of the Invention

[0005] The problem to be solved by this invention is to achieve accurate classification of the growth period of a single tree, and to propose a method, electronic equipment and storage medium for extracting the growth period of a single tree of sparsely planted fruit trees.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for extracting nutrients from a single sparsely planted fruit tree during its growth period includes the following steps:

[0008] S1. Label acquisition: Apply a semantic segmentation model to predict the UAV image p0 of sparsely planted fruit trees, and obtain the canopy vector surface f0 of all fruit trees in p0;

[0009] S2. Label Correction: Combining on-site inspection and visual interpretation of images, manually correct the error prediction of f0 obtained in step S1, and then assign period attributes to obtain the growth period label f. The period attributes include the non-flowering period, flowering period, pruning period, small fruit period and fruit enlargement period.

[0010] S3. Feature Engineering: Calculate texture features for the UAV image p0 of sparsely planted fruit trees collected in step S1, then apply the ReliefF algorithm to select texture features according to weight sorting, and add the selected texture features to the UAV image of sparsely planted fruit trees to obtain the growing season image p.

[0011] S4. Sample combination: Combine the growth period label f ​​obtained in step S2 with the growth period image p obtained in step S3 to form a label-image pair;

[0012] S5. Model Construction: Construct a classification model BDSSNet for individual fruit tree growth stages;

[0013] S6. Model Training: The label-image pairs obtained in step S4 are input into the BDSSNet obtained in step S5 to train the model, resulting in a trained classification model for the growth period of a single fruit tree.

[0014] S7. Predictive Classification: Using the trained fruit tree single-tree growth period classification model obtained in step S6, classify the growth period of sparsely planted fruit trees on the newly predicted single-tree canopy vector surface in the target area.

[0015] Furthermore, in step S1, the semantic segmentation model is selected from UNet, SegNeXt, Deeplabv3, and Deeplabv3+.

[0016] Furthermore, in step S1, the raster obtained by the semantic segmentation model is converted into a vector file in SHP format, and the individual fruit trees in the SHP format vector file are separated to obtain a single canopy vector surface for each fruit tree.

[0017] Furthermore, the specific implementation method of step S3 includes the following steps:

[0018] S3.1. Perform texture feature calculation on the UAV image p0 of sparsely planted fruit trees collected in step S1, including calculating the mean, variance, uniformity, contrast, dissimilarity, entropy, second moment of angle, and correlation to obtain a texture feature map.

[0019] S3.2. Use the ReliefF algorithm to perform feature dimensionality reduction. Generate random points on the texture feature map obtained in step S3.1. Extract the gray value of the region where each point is located as a sample in the ReliefF algorithm. Calculate the weights and then sort the features by importance.

[0020] S3.2.1. Standardize each sample and initialize the weights to 0;

[0021] S3.2.2. Randomly select a sample, use the Euclidean distance method to calculate the distance between the selected sample and other samples, and extract samples of the same class and samples of different classes;

[0022] S3.2.3. Update feature weights based on the distance between samples of the same class and the distance between samples of different classes. The resulting expression is:

[0023]

[0024] in, It is the weight of the j-th feature. These are the selected samples. yes Similar samples, yes Outlier samples, It is a sample and The difference on the j-th feature; It is a sample and The difference on the j-th feature; It is the number of iterations. It is a sample and Probability estimate of belonging to different classes;

[0025] S3.2.4. Repeat steps S3.2.2 and S3.2.3, usually iterating to a fixed number of times or until the algorithm converges. Sort the features according to the final weights, with features having higher weights appearing earlier in the sorting. Obtain the most important features in the texture features and add them to the three-channel UAV image of sparsely planted fruit trees to synthesize a four-channel growing season image p.

[0026] Furthermore, the specific implementation method of step S5 includes the following steps:

[0027] S5.1. Construct an attention mechanism consisting of a combination of grouped cascaded attention and window attention;

[0028] The grouped cascaded attention first constructs an independent query vector Q, key vector K, and value vector V for each attention head. The projected convolutional layer and the depthwise convolutional layer Dw are used to perform depthwise convolution processing on Q. The Proj projection layer is defined to merge the projection layers of the multi-head outputs. At the same time, the relative position encoding is specified, and the relative position offset of all pixel pairs is calculated and indexed to obtain the initial learnable attention bias parameters.

[0029] Window attention is based on grouped cascaded attention and combined with local window judgment. If the input size is less than or equal to the window size, grouped cascaded attention is calculated directly. If the input size is greater than the window size, the input size is padded to a size divisible by the window size, the windows are divided, and then grouped cascaded attention is processed separately for each window. Finally, the windows are merged, the padded size is restored, the padded part is removed, and the original size is adjusted.

[0030] S5.2. Construct the basic module B0, which is divided into a first branch Branch1 and a second branch Branch2. Set Branch1 to remain unchanged. Branch2 is passed through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 5×5 depthwise separable convolution, and a batch normalization layer in sequence. The cloned features are two identical copies. The first copy is passed through the attention module constructed in step S5.1 to obtain attention weights. The attention weights are multiplied with the second copy and then passed through a 1×1 pointwise convolution. Finally, it is connected to Branch1 and passed through a channel shuffling layer to obtain the output of the basic module.

[0031] S5.3. Construct the downsampling module B1; it is divided into a third branch Branch3 and a fourth branch Branch4. Branch3 sequentially passes through a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Branch4 sequentially passes through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Finally, it is connected to Branch3 and passes through a channel shuffling layer to obtain the output of the downsampling module.

[0032] S5.4. Construct the backbone network; the backbone network consists of 5 stages. Stage 1 and Stage 2 are connected by a 3×3 max pooling layer with a stride of 2 and padding of 1. The output of Stage 2 is bilinearly interpolated and concatenated with Stage 3 to obtain feature F2_3. F2_3 is bilinearly interpolated and concatenated with Stage 4 to obtain F2_3_4. F2_3_4 is bilinearly interpolated to obtain feature F2_3_4_5 concatenated with Stage 5. This feature F5, after passing through 5 stages, is fused with the feature fusion layer to obtain the output feature F. Then, a global average pooling layer is applied, and finally, the input is fed into a fully connected layer to obtain the final classification result.

[0033] Stage 1 consists of a 1×1 convolution with a stride of 2 and padding of 1, a batch normalization layer, and a ReLU activation layer;

[0034] Stage 2 consists of one basic module B0 and two downsampling modules B1;

[0035] Stage 3 consists of one basic module B0 and five downsampling modules B1;

[0036] Stage 4 consists of one basic module B0 and two downsampling modules B1;

[0037] Stage 5 consists of a 1×1 convolution with stride of 1 and padding of 0, a batch normalization layer, and a ReLU activation layer.

[0038] The feature fusion layer is a convolutional sequence containing two sets of 3×3 convolutions, a batch normalization layer, a ReLU activation layer, and a regularized Dropout layer. The Dropout ratio in the first set is 0.5, and the Dropout ratio in the second set is 0.1. The feature F obtained by concatenating F2_3_4_5 and F5 is passed through the feature fusion layer.

[0039] Furthermore, in step S5, the loss function is set to the cross-entropy loss function.

[0040] Furthermore, in step S6, the label-image obtained in step S4 is preprocessed before model training. The data preprocessing method combines remote sensing technology to expand the buffer, aiming to moderately buffer the canopy vector of a single fruit tree to alleviate the identification error caused by incomplete canopy coverage of a single tree in the semantic segmentation prediction.

[0041] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for extracting the growth period of a single sparsely planted fruit tree.

[0042] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for extracting the growth period of a single sparsely planted fruit tree.

[0043] The beneficial effects of this invention are:

[0044] This invention discloses a method for extracting the growth period of sparsely planted fruit trees. To enable the model to accurately and efficiently identify the growth period of fruit trees in different regions, a fruit tree growth period dataset is created by manually repairing the canopy layer of individual trees using semantic segmentation. To enrich the feature set of fruit tree growth periods and improve sample separability, eight texture features are calculated and a feature dimensionality reduction method based on the ReliefF algorithm is applied to reduce the dimensionality of texture features in remote sensing images, selectively retaining one-dimensional important features and adding them to the remote sensing data. An attention mechanism is introduced into the basic building blocks to effectively enhance the feature information of fruit tree growth periods. A lightweight network structure BDSSNet, combining remote sensing buffer technology, texture features, and deep and shallow features, is constructed to achieve the organic fusion of deep semantic features, shallow primary features, and optimized texture features, achieving a balance between speed and recognition accuracy with fewer model parameters. Finally, the constructed model is trained, and after obtaining the optimal model, the growth period of individual trees is classified based on the newly predicted canopy vectors of individual trees in the target area. The method of this invention can quickly and accurately identify the growth status of individual fruit trees in an orchard, providing technical support for fruit farmers and insurance companies to manage orchards intelligently. At the same time, the lightweight model structure also provides a theoretical basis and reference value for the deployment of future intelligent orchard systems on mobile and embedded devices. Attached Figure Description

[0045] Figure 1 This is a flowchart of a method for extracting nutrients from a single sparsely planted fruit tree during its growth period, as described in this invention.

[0046] Figure 2 The present invention provides a tag-image pair for different periods, which is displayed by superimposing the growth period tag f and the growth period image p, wherein (a) is the non-flowering period, (b) is the flowering period, (c) is the small fruit period, (d) is the pruning period, and (e) is the swelling period.

[0047] Figure 3 This is a schematic diagram of the buffer expansion of the present invention, wherein (a) is the buffer before expansion, (b) is the buffer after expansion, (c) is the original image of the UAV, and (d) is a comparison before and after expansion;

[0048] Figure 4 This is a block architecture diagram of the inverted residual connection of the present invention, where (a) is the basic module and (b) is the downsampling module;

[0049] Figure 5 This is a diagram of the BDSSNet backbone network architecture of the present invention;

[0050] Figure 6 This is a schematic diagram of the channel washing layer of the present invention;

[0051] Figure 7 This is a schematic diagram of the attention WCG of the present invention;

[0052] Figure 8 This is a graph showing the combination of loss and accuracy during the model training process of this invention.

[0053] Figure 9 This is a diagram showing the results of fruit tree growth period identification in this invention, where (a) is plot 1 and (b) is plot 2;

[0054] Figure 10 Examples of accuracy maps for the combination of modules in the model of this invention are shown below: (a) is the accuracy map without adding a buffer and with the improved backbone network; (b) is the accuracy map with only adding a buffer; and (c) is the accuracy map with adding a buffer and with the improved backbone network. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described specific embodiments are merely a part of the embodiments of the invention, and not all of them. The components of the specific embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations, and the invention may also have other embodiments.

[0056] Therefore, the following detailed description of specific embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected specific embodiments of the invention. All other specific embodiments obtained by those skilled in the art based on these specific embodiments without inventive effort are within the scope of protection of this invention.

[0057] To further understand the invention's content, features, and effects, the following specific embodiments are provided, along with accompanying drawings. Figure 1 - Appendix Figure 10 Detailed explanation is as follows:

[0058] Example 1:

[0059] A method for extracting nutrients from a single sparsely planted fruit tree during its growth period includes the following steps:

[0060] S1. Label acquisition: Apply a semantic segmentation model to predict the UAV image p0 of sparsely planted fruit trees, and obtain the canopy vector surface f0 of all fruit trees in p0;

[0061] Furthermore, in step S1, the raster obtained by the semantic segmentation model is converted into a vector file in SHP format, and the individual fruit trees in the SHP format vector file are separated to obtain a single canopy vector surface for each fruit tree.

[0062] Furthermore, in step S1, the semantic segmentation model is selected from UNet, SegNeXt, Deeplabv3, and Deeplabv3+. This step is only to obtain the canopy vector f0 of a single fruit tree, which facilitates the determination of the growth period at the single-tree level.

[0063] S2. Label Correction: Combining on-site inspection and visual interpretation of images, manually correct the error prediction of f0 obtained in step S1, and then assign period attributes to obtain the growth period label f. The period attributes include the non-flowering period, flowering period, pruning period, small fruit period and fruit enlargement period.

[0064] Furthermore, the specific implementation method of step S2 includes the following steps:

[0065] S2.1 For the single-plant canopy vector f0 from step S1, perform manual correction by removing misdivided other crop vector surfaces, filling in missing single-plant vector surfaces, aggregating over-divided single-plant vector surfaces, peeling off multiple under-divided single-plant vector surfaces, and filling in single-plant vector surfaces with cavities.

[0066] S2.2 For the corrected single-tree vector surface f0, the growth period attribute is assigned by combining the field survey samples at the plot level with the visual interpretation at the single-tree level. The growth period values ​​are divided into five types: "non-flowering period", "flowering period", "pruning period", "small fruit period" and "enlargement period" (including but not limited to, some fruit trees may have fewer than five growth periods, which should be modified according to the actual situation of the fruit tree). The values ​​are input into the attribute table of the vector to obtain the growth period label f.

[0067] S3. Feature Engineering: Calculate texture features for the UAV image p0 of sparsely planted fruit trees collected in step S1, then apply the ReliefF algorithm to select texture features according to weight sorting, and add the selected texture features to the UAV image of sparsely planted fruit trees to obtain the growing season image p.

[0068] Furthermore, the specific implementation method of step S3 includes the following steps:

[0069] S3.1. Perform texture feature calculation on the UAV image p0 of sparsely planted fruit trees collected in step S1, including calculating the mean, variance, uniformity, contrast, dissimilarity, entropy, second moment of angle, and correlation to obtain a texture feature map.

[0070] Furthermore, S3.1.1 Mean represents the average intensity of the image's grayscale values, reflecting the overall brightness level of the image:

[0071] ;

[0072] S3.1.2 Variance is an indicator that measures the dispersion of image grayscale values. The larger the variance, the greater the difference between pixel values, and the higher the image contrast.

[0073] ;

[0074] S3.1.3 Homogeneity, also known as uniformity, describes how similar the gray values ​​of pixel pairs in an image are within their neighborhood. High homogeneity means that the texture in the image is relatively uniform.

[0075] ;

[0076] S3.1.4 Contrast reflects the intensity of local gray-level changes in an image, that is, a measure of the difference between different gray levels. High contrast indicates the presence of obvious edges or details in the image.

[0077] ;

[0078] S3.1.5 Dissimilarity is a variation of contrast that measures the average difference in grayscale values ​​between adjacent pixels in an image, but unlike contrast, it does not emphasize large differences.

[0079] ;

[0080] S3.1.6 Entropy is an information theory concept used to describe the uncertainty or complexity of gray-level distribution in an image. A higher entropy value indicates more chaotic information and more complex texture in the image.

[0081] ;

[0082] S3.1.7 Angular Second Moment (ASM), also known as energy, is a measure of the sum of squares of all elements in the image's gray-level co-occurrence matrix. A larger ASM value indicates more consistent image texture.

[0083] ;

[0084] S3.1.8 Correlation measures the degree of linear correlation between pixels and grayscale values ​​in an image. It can be used to assess the consistency of texture orientation in an image.

[0085] ;

[0086] S3.2. Use the ReliefF algorithm to perform feature dimensionality reduction. Generate random points on the texture feature map obtained in step S3.1. Extract the gray value of the region where each point is located as a sample in the ReliefF algorithm. Calculate the weights and then sort the features by importance.

[0087] S3.2.1. Standardize each sample and initialize the weights to 0;

[0088] S3.2.2. Randomly select a sample, use the Euclidean distance method to calculate the distance between the selected sample and other samples, and extract samples of the same class and samples of different classes;

[0089] S3.2.3. Update feature weights based on the distance between samples of the same class and the distance between samples of different classes. The resulting expression is:

[0090]

[0091] in, It is the weight of the j-th feature. These are the selected samples. yes Similar samples, yes Outlier samples, It is a sample and The difference on the j-th feature; It is a sample and The difference on the j-th feature; It is the number of iterations. It is a sample and Probability estimate of belonging to different classes;

[0092] S3.2.4. Repeat steps S3.2.2 and S3.2.3, usually iterating to a fixed number of times or until the algorithm converges. Sort the features according to the final weights, with features having higher weights appearing earlier in the sorting. Obtain the most important features in the texture features and add them to the three-channel UAV image of sparsely planted fruit trees to synthesize a four-channel growing season image p.

[0093] S4. Sample combination: Combine the growth period label f ​​obtained in step S2 with the growth period image p obtained in step S3 to form a label-image pair;

[0094] Furthermore, in step S4, the tag-image pair has a vector format for the tag and a raster format for the image. The tag vector must include a "period" field, which includes "non-flowering period", "flowering period", "small fruit period", "enlargement period" and "pruning period", and can be modified according to the specific fruit tree's growth period.

[0095] S5. Model Construction: Construct a classification model BDSSNet for individual fruit tree growth stages;

[0096] Furthermore, the specific implementation method of step S5 includes the following steps:

[0097] S5.1. Construct an attention mechanism consisting of a combination of grouped cascaded attention and window attention;

[0098] The grouped cascaded attention first constructs an independent query vector Q, key vector K, and value vector V for each attention head. The projected convolutional layer and the depthwise convolutional layer Dw are used to perform depthwise convolution processing on Q. The Proj projection layer is defined to merge the projection layers of the multi-head outputs. At the same time, the relative position encoding is specified, and the relative position offset of all pixel pairs is calculated and indexed to obtain the initial learnable attention bias parameters.

[0099] Window attention is based on grouped cascaded attention and combined with local window judgment. If the input size is less than or equal to the window size, grouped cascaded attention is calculated directly. If the input size is greater than the window size, the input size is padded to a size divisible by the window size, the windows are divided, and then grouped cascaded attention is processed separately for each window. Finally, the windows are merged, the padded size is restored, the padded part is removed, and the original size is adjusted.

[0100] Furthermore, the Cascaded Group Attention first requires constructing an independent QKV projection convolutional layer and a Dw depth convolutional layer for each attention head.

[0101] During the submodule initialization phase, a QKV layer is defined, creating an independent convolutional BN layer for each attention head to generate a query vector (Q), key vector (K), and value vector (V). A Dw layer is defined, adding a depthwise convolution (enhancing locality) to the query vector (Q) of each head. A Proj projection layer is defined, merging the projection layers of multi-head outputs (including ReLU activation). Simultaneously, relative position encoding is defined, and the relative position offsets of all pixel pairs are calculated and indices are assigned to obtain the initial learnable attention bias parameters.

[0102] In the forward propagation, the input is first processed head-by-head: it is divided into num_heads parts along the channel dimension; then, attention is calculated head-by-head: a cascaded operation, where the input of the current head is the original input plus the output of the previous head, to enhance reuse and progressively refine features, generating Q / K / V, and performing deep convolution on Q to enhance the local context awareness of the query; attention is calculated according to the training or prediction state; finally, weighted aggregation is performed. All heads are concatenated and the output is projected.

[0103] The window attention algorithm is a further operation built upon cascaded group attention. It's a module combining local window partitioning and cascaded group attention, primarily used for efficient attention computation on high-resolution inputs. Its core idea is "divide and conquer": the input feature map is divided into multiple non-overlapping local windows, and attention is computed independently within each window, thus reducing computational complexity. During initialization, the window size validity is checked, ensuring the window size is greater than 0 and not larger than the input size. If the input size is less than or equal to the window size, cascaded group attention is computed directly. If the input size is greater than the window size, the input size is padded to a size divisible by the window size, the windows are partitioned, and then cascaded group attention is processed separately for each window. Finally, the windows are merged, the padded size is restored, the padded portion is removed, and the original size is adjusted.

[0104] Compared to the global attention mechanism, the window attention mechanism reduces computation by nearly 200 times when calculating with a window size of 14×14 and an original size of 28×28. Unlike SwinTransformer, it uses a fixed window instead of a sliding window, resulting in a more lightweight design.

[0105] The Conv2d_BN (Convolution-Batch Normalization) layer combines convolutional layers and batch normalization layers into a sequence module, and merges convolutional layers and BN layers into a single convolutional layer, thereby accelerating inference (reducing computation).

[0106] S5.2. Construct the basic module B0, which is divided into a first branch Branch1 and a second branch Branch2. Set Branch1 to remain unchanged. Branch2 is passed through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 5×5 depthwise separable convolution, and a batch normalization layer in sequence. The cloned features are two identical copies. The first copy is passed through the attention module constructed in step S5.1 to obtain attention weights. The attention weights are multiplied with the second copy and then passed through a 1×1 pointwise convolution. Finally, it is connected to Branch1 and passed through a channel shuffling layer to obtain the output of the basic module.

[0107] The specific expression is:

[0108]

[0109]

[0110]

[0111]

[0112]

[0113] In the formula, dim=1 represents the process along the channel dimension, and chunk(a, b, dim=1) represents dividing tensor a into b equal parts along the channel dimension. Here, the input parameter is to divide the input into two equal parts, namely... and Their sizes are B*C / / 2*H*W respectively. This represents the output of branch 1. This indicates the clonal characteristics of the Branch2 branch. This represents the output of branch 2. Let x represent the output tensor, x represent the input tensor of size B*C*H*W, Conv represent 1×1 pointwise convolution, BN represent batch normalization layer, and ReLU represent ReLU activation layer. This represents a 5×5 depthwise separable convolution. Indicates splicing, 'Att' represents the channel shuffling layer, and 'Att' represents the attention module.

[0114] S5.3. Construct the downsampling module B1; it is divided into a third branch Branch3 and a fourth branch Branch4. Branch3 sequentially passes through a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Branch4 sequentially passes through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Finally, it is connected to Branch3 and passes through a channel shuffling layer to obtain the output of the downsampling module.

[0115] The specific expression is:

[0116]

[0117]

[0118]

[0119]

[0120] In the formula, dim=1 represents the path along the channel dimension, and chunk(a, b, dim=1) represents dividing tensor a into b equal parts along the channel dimension dim=1. Here, the input parameters are the two equal parts that the input is divided into. and Their sizes are B*C / / 2*H*W respectively. This indicates the output of Branch3. This indicates the output of Branch4. Let x' represent the output tensor, x' represent the input tensor of size B*C*H*W, Conv represent a 1×1 pointwise convolution, BN represent a batch normalization layer, and ReLU represent a ReLU activation layer. This represents a 5×5 depthwise separable convolution. Indicates splicing, This indicates the channel washing layer.

[0121] Furthermore, the channel washing layer It includes the following steps:

[0122] ① Reshaping: The reshaping operation divides the channels of the feature map into multiple groups, each containing the same number of channels (groupnum). This is done to prepare for the subsequent transpose operation.

[0123] ② Transpose: By transposing, the channels within a group are rearranged, distributing channels that were originally in the same group to different groups. This step breaks down information silos within a group and promotes full mixing of information.

[0124] ③ Flattening: Finally, the feature map is restored to its original shape through the flattening operation, but the channel order has changed, achieving the effect of channel mixing.

[0125] S5.4. Construct the backbone network; the backbone network consists of 5 stages. Stage 1 and Stage 2 are connected by a 3×3 max pooling layer with a stride of 2 and padding of 1. The output of Stage 2 is bilinearly interpolated and concatenated with Stage 3 to obtain feature F2_3. F2_3 is bilinearly interpolated and concatenated with Stage 4 to obtain F2_3_4. F2_3_4 is bilinearly interpolated to obtain feature F2_3_4_5 concatenated with Stage 5. This feature F5, after passing through 5 stages, is fused with the feature fusion layer to obtain the output feature F. Then, a global average pooling layer is applied, and finally, the input is fed into a fully connected layer to obtain the final classification result.

[0126] The specific expression is:

[0127]

[0128]

[0129]

[0130]

[0131]

[0132]

[0133]

[0134]

[0135]

[0136]

[0137] In the formula Let n represent the feature output in the nth stage, where n ∈ [1, 2, 3, 4, 5]. This represents the output result of concatenating F2 (resampled to size F3) with F3. express The output result after resampling to size F4 and concatenating with F4. express The output after resampling to size F5, where F represents F5 and F5. The output result after the feature fusion layer This indicates the final classification result. This indicates the nth stage of the operation. This indicates max pooling. Indicates will After bilinear interpolation and resampling, Size, Indicates the feature fusion layer. Indicates splicing, Indicates global average pooling. This indicates a fully connected layer.

[0138] Stage 1 consists of a 1×1 convolution with a stride of 2 and padding of 1, a batch normalization layer, and a ReLU activation layer;

[0139]

[0140] In the formula This indicates the output after Stage 1. This indicates input.

[0141] Stage 2 consists of one basic module B0 and two downsampling modules B1;

[0142]

[0143] In the formula This indicates the output after Stage 2. The input is represented by B0, which represents the B0 basic module described in step S4.2. B1 represents the B1 downsampling module described in step S4.2.

[0144] Stage 3 consists of one basic module B0 and five downsampling modules B1;

[0145]

[0146] In the formula This indicates the output after Stage 3. The input is represented by B0, which represents the B0 basic module described in step S4.2. B1 represents the B1 downsampling module described in step S4.2.

[0147] Stage 4 consists of one basic module B0 and two downsampling modules B1;

[0148]

[0149] In the formula This indicates the output after Stage 4. The input is represented by B0, which represents the B0 basic module described in step S4.2. B1 represents the B1 downsampling module described in step S4.2.

[0150] Stage 5 consists of a 1×1 convolution with stride of 1 and padding of 0, a batch normalization layer, and a ReLU activation layer.

[0151]

[0152] In the formula This indicates the output after Stage 5. This indicates input.

[0153] The feature fusion layer is a convolutional sequence containing two sets of 3×3 convolutions, a batch normalization layer, a ReLU activation layer, and a regularized Dropout layer. The Dropout ratio in the first set is 0.5, and the Dropout ratio in the second set is 0.1. The feature F obtained by concatenating F2_3_4_5 and F5 is passed through the feature fusion layer.

[0154]

[0155]

[0156] In the formula This represents the output of the first convolution in the convolution sequence. This represents the second convolution in the convolution sequence, i.e., the output after the feature fusion layer. Indicates input, This represents a 3×3 convolution, BN represents a batch normalization layer, and ReLU represents a ReLU activation layer. , middle represents the regularization layer, and m and n represent the dropout probabilities, where m and n ∈ [0,1).

[0157] Furthermore, in this specific embodiment, within the first group The ratio is 0.5, the second group. The ratio is 0.1.

[0158] Furthermore, in step S5, the loss function is set to the cross-entropy loss function. Step S5 also involves training BDSSNet to obtain the optimal model. In the model evaluation part, the training set loss and validation set loss, along with the overall recognition accuracy, are used during training. For testing, the overall recognition accuracy is used.

[0159]

[0160] Where k is the total number of pixel categories; true positive (TP) represents the number of pixels predicted as true and actually true; false positive (FP) represents the number of pixels predicted as true and actually false; false negative (FN) represents the number of pixels predicted as false and actually true; and true negative (TN) represents the number of pixels predicted as false and actually false.

[0161] S6. Model Training: The label-image pairs obtained in step S4 are input into the BDSSNet obtained in step S5 to train the model, resulting in a trained classification model for the growth period of a single fruit tree.

[0162] Furthermore, in step S6, the label-image obtained in step S4 is preprocessed before model training. The data preprocessing method combines remote sensing technology to expand the buffer, aiming to moderately buffer the canopy vector of a single fruit tree to alleviate the identification error caused by incomplete canopy coverage of a single tree in the semantic segmentation prediction.

[0163] Table 1 shows the software and hardware parameters for model training. The loss curves for the training and validation sets, and the overall recognition accuracy curve are attached. Figure 6 :

[0164] Table 1

[0165]

[0166] S7. Predictive Classification: Using the trained fruit tree single-tree growth period classification model obtained in step S6, classify the growth period of sparsely planted fruit trees on the newly predicted single-tree canopy vector surface in the target area.

[0167] Furthermore, in this embodiment, 280 mango sample plots were randomly selected in the Sanya and Yazhou areas of Hainan Province. The drone data collection time for these plots was from January to November 2024. A semantic segmentation model was applied to invert the canopy of individual trees, followed by format conversion and fragmentation to obtain the vector surface f0 of the canopy of all individual fruit trees. The semantic segmentation method used in this specific embodiment is the mango single-tree extraction method described in the applicant's previously authorized patent number CN119091304B.

[0168] To facilitate understanding of the technical effects of this invention, 10 mango plots in Hainan Province were randomly selected, and partial ablation experiments were conducted using the method described in this embodiment of the invention. Table 2 is the ablation experiment accuracy evaluation table, and appendix... Figure 9 This involves a quantitative accuracy evaluation of the development model of this invention patent.

[0169] Table 2

[0170]

[0171] As can be seen from Table 2, the improved model proposed in this invention has very high overall accuracy in the orchard growth period of the scene image, and shows the best overall performance in category recognition. The combination of modules effectively improves the recognition accuracy.

[0172] Compared with other methods for classifying the growth period of fruit trees, this invention has the following advantages:

[0173] (1) Compared with traditional manual identification, it is more efficient, saves time and effort, and saves costs;

[0174] (2) Compared with simple panoramic semantic segmentation, the identification of the growth period of a single plant is more accurate, more reliable and more granular;

[0175] (3) As a two-stage semantic segmentation + image classification algorithm, it has a simple structure, small number of parameters, and fast training and inference;

[0176] (4) It combines shallow features, texture features, and primary semantic features, and has recognition accuracy;

[0177] (5) Effectively reduces the phenomenon of misidentification during the growth period caused by the small size of the canopy of a single fruit tree in semantic segmentation.

[0178] (6) It solves the problem of intelligent monitoring of the growth period in orchards with high efficiency and high precision;

[0179] In summary, the present invention provides a method for extracting the growth period of sparsely planted fruit trees. To enable the model to accurately and efficiently identify the growth period of individual fruit trees, a fruit tree growth period dataset is created by combining semantic segmentation methods to identify the canopy of individual trees. To enrich the feature set of fruit tree growth periods and improve sample separability, eight texture features are calculated and a feature dimensionality reduction method based on the ReliefF algorithm is applied to achieve dimensionality reduction of texture features in remote sensing images, selectively retaining one-dimensional important features and adding them to the remote sensing data. A WCG attention mechanism is introduced into the basic building blocks, which can effectively enhance the feature information of fruit tree growth periods under a lightweight structure. A lightweight network structure BDSSNet, combining remote sensing buffer technology, texture features, and deep and shallow features, is constructed to achieve the organic integration of deep semantic features, shallow primary features, and optimized texture features, and achieves a balance between speed and recognition accuracy with fewer model parameters. Finally, the constructed model is trained, and after obtaining the optimal model, the growth period of individual fruit trees is classified based on the newly predicted canopy vectors of individual fruit trees in the target area. The method of this invention can quickly and accurately identify the growth status of individual fruit trees in an orchard, providing technical support for fruit farmers and insurance companies to manage fruit trees and orchards intelligently. At the same time, the lightweight model structure also provides a theoretical basis and reference value for the deployment of future intelligent orchard systems on mobile and embedded devices.

[0180] Example 2:

[0181] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the single-tree growth period extraction method for sparsely planted fruit trees described in Embodiment 1.

[0182] The computer device of the present invention may include a processor and a memory, such as a microcontroller containing a central processing unit. Furthermore, the processor executes the computer program stored in the memory to implement the steps of the above-described method for extracting the growth period of a single sparsely planted fruit tree.

[0183] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0184] The memory may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function (such as sound playback, image playback, etc.); the data storage area may store data created based on the use of the mobile phone (such as audio data, phonebook, etc.). Furthermore, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, smart media cards (SMC), secure digital cards (SD cards), flash cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0185] Example 3:

[0186] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for extracting the growth period of a single sparsely planted fruit tree as described in Example 1.

[0187] The computer-readable storage medium of the present invention can be any form of storage medium that can be read by the processor of a computer device, including but not limited to non-volatile memory, volatile memory, ferroelectric memory, etc. The computer-readable storage medium stores a computer program. When the processor of the computer device reads and executes the computer program stored in the memory, the above-mentioned method for extracting the growth period of a single sparsely planted fruit tree can be realized.

[0188] The computer program includes computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0189] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0190] Although this application has been described above with reference to specific embodiments, various modifications can be made and components can be replaced with equivalents without departing from the scope of this application. In particular, as long as there is no structural conflict, the features in the specific embodiments disclosed in this application can be combined with each other in any way. The lack of an exhaustive description of these combinations in this specification is merely for the sake of brevity and resource conservation. Therefore, this application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for extracting nutrients from a single sparsely planted fruit tree during its growth period, characterized in that, Includes the following steps: S1. Label acquisition: Apply a semantic segmentation model to predict the UAV image p0 of sparsely planted fruit trees, and obtain the canopy vector surface f0 of all fruit trees in p0; S2. Label Correction: Combining on-site inspection and visual interpretation of images, manually correct the error prediction of f0 obtained in step S1, and then assign period attributes to obtain the growth period label f. The period attributes include the non-flowering period, flowering period, pruning period, small fruit period and fruit enlargement period. S3. Feature Engineering: Calculate texture features for the UAV image p0 of sparsely planted fruit trees collected in step S1, then apply the ReliefF algorithm to select texture features according to weight sorting, and add the selected texture features to the UAV image of sparsely planted fruit trees to obtain the growing season image p. S4. Sample combination: Combine the growth period label f ​​obtained in step S2 with the growth period image p obtained in step S3 to form a label-image pair; S5. Model Construction: Construct a classification model BDSSNet for individual fruit tree growth stages; The specific implementation method of step S5 includes the following steps: S5.

1. Construct an attention mechanism consisting of a combination of grouped cascaded attention and window attention; The grouped cascaded attention first constructs an independent query vector Q, key vector K, and value vector V for each attention head. The projected convolutional layer and the depthwise convolutional layer Dw are used to perform depthwise convolution processing on Q. The Proj projection layer is defined to merge the projection layers of the multi-head outputs. At the same time, the relative position encoding is specified, and the relative position offset of all pixel pairs is calculated and indexed to obtain the initial learnable attention bias parameters. Window attention is based on grouped cascaded attention and combined with local window judgment. If the input size is less than or equal to the window size, grouped cascaded attention is calculated directly. If the input size is greater than the window size, the input size is padded to a size divisible by the window size, the windows are divided, and then grouped cascaded attention is processed separately for each window. Finally, the windows are merged, the padded size is restored, the padded part is removed, and the original size is adjusted. S5.

2. Construct the basic module B0, which is divided into a first branch Branch1 and a second branch Branch2. Set Branch1 to remain unchanged. Branch2 is passed through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 5×5 depthwise separable convolution, and a batch normalization layer in sequence. The cloned features are two identical copies. The first copy is passed through the attention module constructed in step S5.1 to obtain attention weights. The attention weights are multiplied with the second copy and then passed through a 1×1 pointwise convolution. Finally, it is connected to Branch1 and passed through a channel shuffling layer to obtain the output of the basic module. S5.

3. Construct the downsampling module B1; it is divided into a third branch Branch3 and a fourth branch Branch4. Branch3 sequentially passes through a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Branch4 sequentially passes through a 1×1 pointwise convolution, a batch normalization layer, a ReLU activation layer, a 3×3 depthwise separable convolution, a batch normalization layer, a 1×1 pointwise convolution, a batch normalization layer, and a ReLU activation layer. Finally, it is connected to Branch3 and passes through a channel shuffling layer to obtain the output of the downsampling module. S5.

4. Construct the backbone network; the backbone network consists of 5 stages. Stage 1 and Stage 2 are connected by a 3×3 max pooling layer with a stride of 2 and padding of 1. The output of Stage 2 is bilinearly interpolated and concatenated with Stage 3 to obtain feature F2_3. F2_3 is bilinearly interpolated and concatenated with Stage 4 to obtain F2_3_4. F2_3_4 is bilinearly interpolated to obtain feature F2_3_4_5 concatenated with Stage 5. This feature F5, after passing through 5 stages, is fused with the feature fusion layer to obtain the output feature F. Then, a global average pooling layer is applied, and finally, the input is fed into a fully connected layer to obtain the final classification result. Stage 1 consists of a 1×1 convolution with a stride of 2 and padding of 1, a batch normalization layer, and a ReLU activation layer; Stage 2 consists of one basic module B0 and two downsampling modules B1; Stage 3 consists of one basic module B0 and five downsampling modules B1; Stage 4 consists of one basic module B0 and two downsampling modules B1; Stage 5 consists of a 1×1 convolution with stride of 1 and padding of 0, a batch normalization layer, and a ReLU activation layer. The feature fusion layer is a convolutional sequence containing two sets of 3×3 convolutions, a batch normalization layer, a ReLU activation layer, and a regularized Dropout layer. The Dropout ratio in the first set is 0.5, and the Dropout ratio in the second set is 0.

1. The feature F obtained by concatenating F2_3_4_5 and F5 is passed through the feature fusion layer. S6. Model Training: The label-image pairs obtained in step S4 are input into the BDSSNet obtained in step S5 to train the model, resulting in a trained classification model for the growth period of a single fruit tree. S7. Predictive Classification: Using the trained fruit tree single-tree growth period classification model obtained in step S6, classify the growth period of sparsely planted fruit trees on the newly predicted single-tree canopy vector surface in the target area.

2. The method for extracting nutrients from a single sparsely planted fruit tree during its growth period according to claim 1, characterized in that, In step S1, the semantic segmentation model is selected from UNet, SegNeXt, Deeplabv3, and Deeplabv3+.

3. A method for extracting nutrients from a single sparsely planted fruit tree during its growth period, as described in claim 1 or 2, characterized in that... In step S1, the raster predicted by the semantic segmentation model is converted into a vector file in SHP format, and the individual fruit trees in the SHP format vector file are separated to obtain a single canopy vector surface for each fruit tree.

4. The method for extracting nutrients from a single sparsely planted fruit tree during its growth period according to claim 3, characterized in that, The specific implementation method of step S3 includes the following steps: S3.

1. Perform texture feature calculation on the UAV image p0 of sparsely planted fruit trees collected in step S1, including calculating the mean, variance, uniformity, contrast, dissimilarity, entropy, second moment of angle, and correlation to obtain a texture feature map. S3.

2. Use the ReliefF algorithm to perform feature dimensionality reduction. Generate random points on the texture feature map obtained in step S3.

1. Extract the gray value of the region where each point is located as a sample in the ReliefF algorithm. Calculate the weights and then sort the features by importance. S3.2.

1. Standardize each sample and initialize the weights to 0; S3.2.

2. Randomly select a sample, use the Euclidean distance method to calculate the distance between the selected sample and other samples, and extract samples of the same class and samples of different classes; S3.2.

3. Update feature weights based on the distance between samples of the same class and the distance between samples of different classes. The resulting expression is: ； in, It is the weight of the j-th feature. These are the selected samples. yes Similar samples, yes Outlier samples, It is a sample and The difference on the j-th feature; It is a sample and The difference on the j-th feature; It is the number of iterations. It is a sample and Probability estimate of belonging to different classes; S3.2.

4. Repeat steps S3.2.2 and S3.2.3, usually iterating to a fixed number of times or until the algorithm converges. Sort the features according to the final weights, with features having higher weights appearing earlier in the sorting. Obtain the most important features in the texture features and add them to the three-channel UAV image of sparsely planted fruit trees to synthesize a four-channel growing season image p.

5. The method for extracting nutrients from a single sparsely planted fruit tree during its growth period according to claim 4, characterized in that, In step S5, the loss function is set to the cross-entropy loss function.

6. The method for extracting nutrients from a single sparsely planted fruit tree during its growth period according to claim 5, characterized in that, In step S6, the label-image obtained in step S4 is preprocessed before model training. The data preprocessing method combines remote sensing technology to expand the buffer. The purpose is to moderately buffer the canopy vector of a single fruit tree to alleviate the identification error caused by incomplete canopy coverage of a single tree in the semantic segmentation prediction.

7. An electronic device, characterized in that, It includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for extracting the growth period of a single sparsely planted fruit tree as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for extracting the growth period of a single sparsely planted fruit tree as described in any one of claims 1-6.

Images