A rice yield prediction method based on double-branch context fusion

Abstract

The application provides a rice yield prediction method based on a double-branch context fusion, comprising: obtaining an original rice image to be processed; pre-processing the original rice image to obtain a first image; inputting the first image into a trained yield prediction model to obtain a yield prediction value, wherein the yield prediction model comprises: an image block embedding layer, used for block processing and linear projection processing on the first image to obtain a first feature after dimension reduction; a plurality of stacked feature extraction blocks, used for extracting and fusing multi-scale context features from the first feature to obtain a second feature; and a regression head, used for mapping the second feature into the yield prediction value. The double-branch network is used for respectively extracting spike texture and plant spatial distribution features and adaptively fusing them, so that the shortcomings of traditional models in modeling microstructure and macro mode are overcome, and the yield prediction precision and generalization ability are significantly improved, thereby providing reliable support for precision agriculture.

Description

Technical Field

[0001] This invention relates to the fields of agricultural remote sensing and artificial intelligence, and in particular to a method for predicting rice yield based on dual-branch context fusion. Background Technology

[0002] Rice is one of the world's most important food crops, providing staple food for more than half of the global population. Its stable supply is directly related to national food security, social stability, and economic development. With increasingly scarce global arable land resources and continuous population growth, accurate, dynamic, and non-destructive prediction of rice yield has become a key technological support for modern intelligent agricultural management, the construction of food security early warning systems, and the optimization of agronomic measures. Traditional rice yield assessment relies heavily on manual field sampling and actual yield measurement, which is labor-intensive and costly, and is becoming increasingly inadequate for large-scale, high-frequency monitoring scenarios, necessitating a shift towards automation and intelligentization.

[0003] In recent years, UAV multispectral remote sensing technology has become an important tool for field crop phenotypic monitoring and yield prediction due to its advantages such as flexibility, centimeter-level resolution, and ability to acquire multi-band spectral information. Current mainstream yield prediction methods are mainly divided into two categories: one is statistical regression models based on vegetation indices (such as NDVI and EVI), combined with machine learning algorithms such as random forests and support vector machines to establish yield prediction relationships. However, these methods heavily rely on manually designed features, have limited feature dimensions, and struggle to fully extract the deep structural information contained in the images. The other category is deep learning image regression models, such as ConvNeXt and DenseNet convolutional neural networks, which can automatically extract multi-level image features and are superior to traditional vegetation index methods to some extent. However, deep learning models generally suffer from insufficient capture of ear structure details, weak modeling of plant spatial distribution patterns, and coarse multi-scale feature fusion methods lacking adaptive collaborative mechanisms, resulting in limited prediction accuracy and generalization ability. Summary of the Invention

[0004] In view of the above-mentioned deficiencies of the prior art, the present invention provides a rice yield prediction method based on dual-branch context fusion to solve the technical problems of insufficient extraction of panicle details and plant spatial distribution features by existing models and the limited prediction accuracy caused by the crude fusion mechanism.

[0005] To achieve the above and other related objectives, this invention provides a rice yield prediction method based on dual-branch context fusion, comprising: acquiring an original rice image to be processed; preprocessing the original rice image to obtain a first image; inputting the first image into a trained yield prediction model to obtain a predicted yield value, wherein the yield prediction model comprises: an image patch embedding layer for performing block segmentation and linear projection processing on the first image to obtain a dimensionality-reduced first feature; multiple stacked feature extraction blocks for extracting and fusing multi-scale contextual features from the first feature to obtain a second feature; and a regression head for mapping the second feature to the predicted yield value.

[0006] In one embodiment of the present invention, the expression of the feature extraction block is as follows:

[0007] {X1,X2}=Split(X in1 ),

[0008] X3=H FFM (H DEM (X1)+H CIM (X2)),

[0009] X out1 =AvgPool(LayerNorm(X3+X in1 )),

[0010] In the formula, X in1 X out1 X1 and X3 are the input and output features of the feature extraction block, respectively; Split is the splitting operation, LayerNorm is the layer normalization operation, and AvgPool is the average pooling operation; H DEM H CIM H FFM These are the detail enhancement module, the context aggregation module, and the feature fusion module.

[0011] In one embodiment of the present invention, the expression of the detail enhancement module is as follows:

[0012] X4=BN(Conv 3×3 (X in2 )),

[0013] X5=ReLU(BN(Conv 3×3 (ReLU(X4)))+X4),

[0014] X6=BN(Conv 5×5 (X5)),

[0015] X out2 =ReLU(BN(Conv5×5 (ReLU(X6)))+X6),

[0016] In the formula, X in2 X out2 X4 and X6 are the input and output features of the detail enhancement module, respectively; Conv is the convolution operation; BN is the batch normalization layer; and ReLU is the activation function.

[0017] In one embodiment of the present invention, the expression of the context aggregation module is as follows:

[0018] X7=LayerNorm(X in3 ),

[0019] X8 = SiLU(Linear(X7)),

[0020] X9=LayerNorm(SS2D(SiLU(DWConv 3×3 (Linear(X7))))),

[0021] X out3 =Linear(X8⊙X9),

[0022] In the formula, X in3 X out3 X7~X9 are the input and output features of the context aggregation module, respectively; Linear is the linear layer; SiLU is the activation function; DWConv is the depthwise separable convolution; SS2D is the two-dimensional selective scanning unit; and ⊙ is the element-wise multiplication.

[0023] In one embodiment of the present invention, the two-dimensional selective scanning unit processes the input features according to the following steps: the input features of the two-dimensional selective scanning unit are scanned and expanded along four different preset scanning directions to obtain four one-dimensional feature sequences; for each one-dimensional feature sequence, a parameter-dependent selective state space model is used to perform sequence modeling to obtain a context-enhanced one-dimensional output sequence; the one-dimensional output sequence is scanned and merged along the opposite direction to the preset scanning direction to restore a feature map with the same dimension as the input features of the two-dimensional selective scanning unit; the four feature maps are fused to obtain the output features of the two-dimensional selective scanning unit.

[0024] In one embodiment of the present invention, for each of the one-dimensional feature sequences, a parameter-dependent selective state-space model is used to perform sequence modeling to obtain a context-enhanced one-dimensional output sequence, including: generating a time step based on the one-dimensional feature sequence. Parameters B and C; based on the time step The parameters B and the preset parameter A are used to obtain the parameters for recursive calculation. and parameters Based on the parameters ,parameter The one-dimensional feature sequence is recursively calculated using parameter C and preset parameter D to generate the one-dimensional output sequence.

[0025] In one embodiment of the present invention, the feature fusion module processes the input features according to the following steps: The output features of the detail enhancement module and the context aggregation module are subjected to adaptive weight vector generation processing to obtain a first weight factor and a second weight factor corresponding to the two output features, respectively; the output features of the detail enhancement module are weighted using the first weight factor, and the output features of the context aggregation module are weighted using the second weight factor; the two weighted features are concatenated by channels to obtain a first fused feature; the first fused feature is sequentially subjected to pointwise convolution and GeLU activation to generate a gating signal; the first fused feature is sequentially subjected to pointwise convolution, 3×3 depthwise separable convolution, and 5×5 depthwise separable convolution to obtain a multi-scale spatial feature; the gating signal and the multi-scale spatial feature are subjected to element-wise multiplication to obtain a second fused feature; the second fused feature is sequentially subjected to pointwise convolution and layer normalization processing to obtain the output feature of the feature fusion module.

[0026] In one embodiment of the present invention, the output features of the detail enhancement module and the context aggregation module are subjected to adaptive weight vector generation processing to obtain a first weight factor and a second weight factor corresponding to the two output features, respectively. This includes: concatenating the output features of the detail enhancement module and the context aggregation module along the channel dimension to obtain a third fusion feature; performing global average pooling processing on the third fusion feature to obtain a fusion vector that summarizes the information of the two branches; and activating the fusion vector through a linear layer and a scaled Sigmoid to generate the first weight factor and the second weight factor.

[0027] In one embodiment of the present invention, the fusion vector is activated by a linear layer and a scaled Sigmoid to generate the first weight factor and the second weight factor, including: activating the fusion vector by a linear layer and a scaled Sigmoid to generate the first weight factor and the second weight factor in the range [0.8, 1.2].

[0028] In one embodiment of the present invention, the number of stacked feature extraction blocks is equal to 10.

[0029] The beneficial effects of this invention are as follows: This invention proposes a rice yield prediction method based on dual-branch context fusion. This method constructs a dual-branch context fusion network, which utilizes a detail enhancement module to enhance the extraction of high-frequency texture and morphological features of rice panicles, and a context integration module to model the spatial distribution and long-distance dependencies between plants. Then, a feature fusion module is used to achieve adaptive collaborative fusion of cross-scale information. It effectively overcomes the problems of traditional models' insufficient capture of micro-panicle structure and weak modeling of macro-distribution patterns, significantly improving the accuracy and generalization ability of yield prediction, and providing reliable technical support for precision agriculture. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0031] Figure 1 A flowchart of a rice yield prediction method provided in an embodiment of the present invention;

[0032] Figure 2 This is an architecture diagram of a production prediction model provided in an embodiment of the present invention;

[0033] Figure 3 An architecture diagram of a detail enhancement module provided in an embodiment of the present invention;

[0034] Figure 4 This is an architecture diagram of a context aggregation module provided in an embodiment of the present invention;

[0035] Figure 5 This is an architectural diagram of a two-dimensional selective scanning unit provided in an embodiment of the present invention;

[0036] Figure 6 This is an architecture diagram of a feature fusion module provided in an embodiment of the present invention. Detailed Implementation

[0037] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0038] It should be understood that the terminology used in the embodiments of this invention is for describing specific particular implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0039] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0040] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0041] Please see Figure 1 , Figure 1 An embodiment of the present invention provides a method for predicting rice yield based on dual-branch context fusion, comprising the following steps.

[0042] 1. Acquire raw rice images to be processed. Raw rice images are pictures or videos of rice captured by image acquisition devices (such as cameras or video cameras), where videos can be processed into images. For example, during the rice ripening period, drones can be used to acquire raw rice images. When acquiring images, it is best to conduct the flight operation during clear, cloudless days with a stable solar altitude angle. This ensures the quality of the acquired images and lays the foundation for further processing. For example, the drone can fly at an altitude of 80 meters to acquire images in five bands: blue (450nm), green (560nm), red (650nm), red edge (730nm), and near-infrared (840nm) as raw rice images.

[0043] 2. Preprocess the original rice image to obtain the first image.

[0044] During the model training phase, preprocessing specifically involved: using Agisoft Metashape software to perform radiometric correction on the raw rice images (a type of multispectral imagery) captured by the drone, constructing point clouds, generating digital surface models, and stitching orthophotos to obtain complete orthophotos of the fields. Subsequently, in ArcMap software, the ROI tool was used to crop each field into three non-overlapping sub-images, generating multiple samples. Then, all sub-images were uniformly scaled to the standard size specified by the model (e.g., 224×224 pixels) through a resampling operation. Finally, these processed sub-images were used as the first images for training.

[0045] During the model inference and prediction phase, preprocessing does not require cropping of sub-images within the field. Only the original rice image to be predicted needs to undergo the same radiometric correction and orthophoto stitching process to generate a complete orthophoto of the field to be predicted. Subsequently, this complete field image is scaled to the same standard size as during training and directly input into the trained yield prediction model as the first image for inference.

[0046] Understandably, the first image used for training, along with the measured yield data of the corresponding field area, can serve as a sample. When the number of samples is insufficient, data augmentation strategies can be employed, including geometric transformations such as random cropping, translation, and flipping, to increase the number of training samples and the model's generalization ability. Finally, the samples are divided into training and test sets in a 4:1 ratio.

[0047] 3. Input the first image into the trained yield prediction model to obtain the predicted yield value. Use the training set above to train the yield prediction model ( Figure 1 The dense context fusion network model (which is the untrained yield prediction model) is trained. After training, the first image used for inference is directly input into the model to obtain the corresponding yield prediction value.

[0048] Please see Figure 2 In a specific embodiment of the present invention, the yield prediction model (i.e., the dense context fusion network model) includes an image patch embedding layer, multiple stacked feature extraction blocks, and a regression head. The image patch embedding layer is used to perform block segmentation and linear projection processing on a first image to obtain a dimensionality-reduced first feature; the multiple stacked feature extraction blocks are used to extract and fuse multi-scale contextual features from the first feature to obtain a second feature; and the regression head is used to map the second feature to a yield prediction value.

[0049] The image patch embedding layer first takes the input image x∈R H×W×C The image is divided into non-overlapping 4×4 blocks to form a one-dimensional sequence. Then, each image block is mapped to a feature space of dimension C through linear projection to generate a feature map of size (H / 4)×(W / 4)×C, thereby reducing computational complexity and preserving the original spatial structure information, while facilitating subsequent serialization processing.

[0050] In one specific embodiment of the present invention, the number of stacked feature extraction blocks can be, for example, 10. The number of stacks determines the depth and receptive field of the network. Too few stacks will lead to insufficient feature extraction, while too many stacks will easily cause gradient vanishing and overfitting. Typically, this number is optimized between 4 and 16 layers based on the complexity of the dataset and computational resources. In this embodiment, the 10-layer design achieves the best balance between model performance and complexity.

[0051] Please see Figure 2 In a specific embodiment of the present invention, the expression of the feature extraction block is as follows:

[0052] {X1,X2}=Split(X in1 ),

[0053] X3=H FFM (H DEM (X1)+H CIM (X2)),

[0054] X out1 =AvgPool(LayerNorm(X3+X in1 )),

[0055] In the formula, X in1 X out1 X1 and X3 are the input and output features of the feature extraction block, respectively; Split is the splitting operation, LayerNorm is the layer normalization operation, and AvgPool is the average pooling operation; H DEM H CIM H FFMThese are the Detail Enhancement Module (DEM), Contextual Integration Module (CIM), and Feature Fusion Module (FFM), respectively. In this embodiment, the input feature X is denoted as... in1 The input features, with dimensions H1×W1×C1, are first divided in half along the channel dimension using a segmentation operation, resulting in features X1 and X2, both with dimensions H1×W1×(C1 / 2). X1 is input to the detail enhancement module to strengthen the spikelet structure, and X2 is input to the context aggregation module to model plant distribution patterns. Subsequently, the feature fusion module adaptively merges the outputs of the two branches and performs cross-scale interactive fusion to generate a unified fused feature map with dimensions H1×W1×C1. This fused feature map is then added back to the original input feature X1 through residual connections. in1 To mitigate gradient vanishing and preserve low-level features, LayerNorm operation is used to stabilize training, and Avgpool is used to progressively downsample the feature map to a size of (H1 / 2)×(W1 / 2)×C1, achieving efficient multi-resolution representation learning. After stacking N blocks, the feature map is progressively compressed to a low-resolution global representation, providing rich contextual information for the regression head.

[0056] Please see Figure 3 In a specific embodiment of the present invention, the expression of the detail enhancement module is as follows:

[0057] X4=BN(Conv 3×3 (X in2 )),

[0058] X5=ReLU(BN(Conv 3×3 (ReLU(X4)))+X4),

[0059] X6=BN(Conv 5×5 (X5)),

[0060] X out2 =ReLU(BN(Conv 5×5 (ReLU(X6)))+X6),

[0061] In the formula, X in2 X out2 X4 and X6 are the input and output features of the detail enhancement module, respectively; Conv is the convolution operation, BN is the batch normalization layer, and ReLU is the activation function. The detail enhancement module aims to strengthen the structural features of the ear by cascading two residual blocks of different scales to achieve progressive structural detail extraction.

[0062] The process of the first residual block is as follows: Input feature X in2 The first input is a residual block, which uses a 3×3 convolutional kernel and focuses on extracting basic edges and local structural contours. Specifically, the input feature X... in2 After initial feature transformation via 3×3 convolution, a batch normalization (BN) layer standardizes the feature distribution to accelerate training and mitigate internal covariate bias. Then, a ReLU activation function introduces non-linearity to enhance the model's expressive power. The output is further fed into a second 3×3 convolution and batch normalization layer for finer local detail sharpening. Subsequently, a skip connection adds the path output to the intermediate feature X4, forming a residual structure that effectively alleviates the vanishing gradient problem in deep networks while preserving shallow detail information. Finally, a ReLU activation layer activates the fused result, generating an intermediate feature map.

[0063] The output of the first residual block serves as the input to the second residual block, which uses a 5×5 convolutional kernel to capture the larger structure of the rice panicle. The specific process is similar: the input undergoes initial feature extraction by passing through a 5×5 convolution and batch normalization layer to expand the receptive field; ReLU activation introduces non-linearity; then, a second 5×5 convolution and batch normalization layer further amplifies high-frequency attributes and sharpens boundaries; skip connections are used to add the intermediate features (X6) to prevent information loss; finally, ReLU activation outputs enhanced detailed features.

[0064] The output of the entire detail enhancement module is an enhanced feature map X processed by two cascaded residual blocks. out2 The residual structure ensures gradual accumulation of details, improving the discriminative power of high-frequency ear attributes and the model's robustness to micro-variations. This module achieves efficient enhancement of small ear fragments to medium-to-large ear structures through convolutional cascading, avoiding the fragmentation problem of traditional single-scale convolution.

[0065] Please see Figure 4 In a specific embodiment of the present invention, the expression of the context aggregation module is as follows:

[0066] X7=LayerNorm(X in3 ),

[0067] X8 = SiLU(Linear(X7)),

[0068] X9=LayerNorm(SS2D(SiLU(DWConv 3×3 (Linear(X7))))),

[0069] X out3 =Linear(X8⊙X9),

[0070] In the formula, X in3 X out3 X7~X9 are the input and output features of the context aggregation module, respectively; Linear is the linear layer; SiLU is the activation function; DWConv is the depthwise separable convolution; SS2D is the two-dimensional selective scanning unit; and ⊙ represents element-wise multiplication. The context aggregation module aims to model the spatial distribution pattern of plants. Through multi-directional scanning of spatial paths, selective state space serialization, and dynamic modulation mechanism of gated paths, it achieves efficient long-distance dependency capture and global context integration.

[0071] In this embodiment, the input feature X in3 First, the data is normalized using a LayerNorm layer to mitigate internal covariate bias, stabilize gradient flow, and improve training convergence speed. The normalized features are then used as input for the subsequent bi-branch.

[0072] The standardized feature X7 is simultaneously split into two parallel paths: the SpatialPath (spatial modeling path) and the GatePath (gated path). The SpatialPath first performs dimensionality projection through a linear layer to adjust the channel dimensions; subsequently, it applies a 3×3 depthwise separable convolution (DWConv). 3×3 This operation first performs depthwise convolution to extract local spatial features, then performs pointwise convolution to fuse channel information, efficiently capturing edge and texture details while maintaining low computational complexity. Next, the SiLU activation function introduces non-linearity, which is smoother than ReLU and avoids gradient vanishing. The activated features are then fed into a two-dimensional selective scan unit (SS2D) for global sequence modeling and long-range dependency extraction. Finally, this path passes through another layer normalization (LayerNorm) layer to further normalize the output features and ensure numerical stability. In the gating path, a modulated signal is generated through linear transformation and SiLU activation. This path independently captures the semantic gating information of the input, used to dynamically adjust the weights of the spatial path features and avoid irrelevant noise interference.

[0073] The outputs of the spatial modeling path and the gated path are multiplied element-wise to achieve feature modulation, enhancing relevant context and suppressing irrelevant parts. The final feature map X is then integrated through a linear layer. out3 This output captures the global dependence of plant spatial distribution, improving the model's robustness to non-uniform field layouts.

[0074] Please see Figure 5 In a specific embodiment of the present invention, the two-dimensional selective scanning unit processes the input features according to the following steps 5.1 to 5.4.

[0075] Step 5.1: The input features of the two-dimensional selective scanning unit are scanned and expanded along four different preset scanning directions to obtain four one-dimensional feature sequences. The scanning expansion serializes the two-dimensional feature map (the input features of the two-dimensional selective scanning unit in this invention are three-dimensional, and the first two dimensions are scanned and expanded) into a one-dimensional sequence in four directions (top left → bottom right, top right → bottom left, bottom left → top right, and bottom right → top left) to cover the entire spatial path of the image and avoid the bias of unidirectional scanning.

[0076] Step 5.2: For each one-dimensional feature sequence, use a parameter-dependent selective state-space model to perform sequence modeling, and obtain a context-enhanced one-dimensional output sequence.

[0077] In a specific embodiment of the present invention, step 5.2 includes steps 5.2.1 to 5.2.3.

[0078] Step 5.2.1: Generate the time step based on the one-dimensional feature sequence. Parameters B and C. The formulas for calculating these three parameters are as follows:

[0079] ,

[0080] ,

[0081] ,

[0082] In the formula, Linear1, Linear2, and Linear3 are three independent learnable linear projection layers, x t is the input value of the one-dimensional feature sequence currently being processed at time step t, and bias is the learnable bias term of the linear projection layer Linear1.

[0083] Step 5.2.2: Based on the time step Parameter B and the preset parameter A are used to obtain the parameters for recursive calculation. and parameters A is the state matrix, and the parameters are... and parameters The calculation formula is as follows:

[0084] ,

[0085] ,

[0086] In the formula, I is the identity matrix, and exp is the matrix exponential function. For matrix The inverse of this process is called zero-order preserved discretization, which transforms continuous state-space parameters into parameters suitable for recursive computation of discrete-time series.

[0087] Step 5.2.3, based on parameters ,parameter The one-dimensional feature sequence is recursively calculated using parameters C and a preset parameter D to generate a one-dimensional output sequence. The continuous state-space model is defined as follows:

[0088] ,

[0089] ,

[0090] In the formula, h t Let y represent the hidden state at time step t. t This is the output sequence at time step t. The one-dimensional output sequence consists of the outputs at all time steps {y1, y2, ..., y...}. L The sequence is composed of}, where L is the length of the one-dimensional feature sequence.

[0091] Step 5.3: Scan and merge the one-dimensional output sequence in the opposite direction to the preset scanning direction to restore it to a feature map with the same input feature dimension as the two-dimensional selective scanning unit.

[0092] Step 5.4: Fuse the four feature maps (usually by adding or concatenating them and then convolving) to obtain the output features of the two-dimensional selective scanning unit.

[0093] Please see Figure 6 In a specific embodiment of the present invention, the feature fusion module processes the input features according to the following steps: (1) The output features of the detail enhancement module and the context aggregation module are subjected to adaptive weight vector generation processing to obtain the first weight factor and the second weight factor corresponding to the two output features respectively. (2) The output features of the detail enhancement module are weighted using the first weight factor, and the output features of the context aggregation module are weighted using the second weight factor. The two weighted features are then concatenated by channels to obtain the first fused feature; which can be expressed by the formula: X CIM1 =W a ×X out2 +W b ×X out3 (3) Perform pointwise convolution and GeLU activation on the first fused feature sequentially to generate a gated signal. (4) Perform pointwise convolution, 3×3 depthwise separable convolution and 5×5 depthwise separable convolution on the first fused feature sequentially to obtain multi-scale spatial features. (5) Perform element-wise multiplication on the gated signal and multi-scale spatial features to obtain the second fused feature. (6) Perform pointwise convolution and layer normalization on the second fused feature sequentially to obtain the output features of the feature fusion module.

[0094] In a specific embodiment of the present invention, the calculation process of the two weight factors includes: (1) concatenating the output features of the detail enhancement module and the context aggregation module in the channel dimension to obtain a third fusion feature; (2) performing global average pooling (GAP) on the third fusion feature to obtain a fusion vector that summarizes the information of the two branches; (3) activating the fusion vector through a linear layer and a scaled Sigmoid to generate the first weight factor W. a Second weighting factor W b .

[0095] In a specific embodiment of the present invention, the first weighting factor W a Second weighting factor W b The value range is [0.8, 1.2]. This range is designed to make minor adjustments to the features, which can achieve adaptive enhancement or suppression, while avoiding gradient instability or training oscillation caused by extreme weights. It is achieved through a scaled sigmoid activation function, which is dynamically shifted slightly around 1.0, thereby enabling fine-grained control over the importance of features.

[0096] The regression head can, for example, consist of a global average pooling layer and a fully connected layer. After stacking N feature extraction blocks, the final feature map first passes through a global average pooling layer, which aggregates the spatial features of each channel into a scalar, forming a one-dimensional global feature vector. Subsequently, this feature vector is fed into a fully connected layer, which directly maps the high-dimensional feature vector into a single, continuous numerical value, i.e., the final predicted rice yield. ).

[0097] It should be noted that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0098] When training a model using a training set, the trained model can be evaluated using metrics such as the coefficient of determination (R²), root mean square error (RMSE), and mean absolute error (MAE). Only models that meet the threshold values ​​set by these metrics are considered the final trained yield prediction model.

[0099] The yield prediction model in this invention is compared with some common yield regression prediction methods in recent years (EfficientnetV2, ConvnextV2, Densenet, Edgenext and RepVgg). The comparison results are shown in Table 1, where the best result is indicated in bold.

[0100] Table 1: Comparison of Rice Yield Prediction Results Using Different Network Models

[0101]

[0102] As can be seen from the table above, this invention effectively extracts the structure of the rice panicle and the distribution of the plant through an innovative dual-branch context fusion network, and further integrates the two in a synergistic manner, achieving the best rice yield prediction performance.

[0103] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A rice yield prediction method based on dual-branch context fusion, characterized in that, include: Obtain the original rice image to be processed; The original rice image is preprocessed to obtain the first image; The first image is input into a trained yield prediction model to obtain a yield prediction value, wherein the yield prediction model includes: An image patch embedding layer is used to perform block segmentation and linear projection processing on the first image to obtain the first feature after dimensionality reduction. Multiple stacked feature extraction blocks are used to extract and fuse multi-scale contextual features from the first feature to obtain the second feature; A regression head is used to map the second feature to the predicted yield value; The expression for the feature extraction block is as follows: {X1,X2}=Split(X in1 ), X3=H FFM (H DEM (X1)+H CIM (X2)), X out1 =AvgPool(LayerNorm(X3+X in1 )), In the formula, X in1 X out1 X1 and X3 are the input and output features of the feature extraction block, respectively; Split is the splitting operation, LayerNorm is the layer normalization operation, and AvgPool is the average pooling operation; H DEM H CIM H FFM These are the detail enhancement module, the context aggregation module, and the feature fusion module.

2. The rice yield prediction method based on dual-branch context fusion according to claim 1, characterized in that, The expression for the detail enhancement module is as follows: X4=BN(Conv 3×3 (X in2 )), X5=ReLU(BN(Conv 3×3 (ReLU(X4)))+X4), X6=BN(Conv 5×5 (X5)), X out2 =ReLU(BN(Conv 5×5 (ReLU(X6)))+X6), In the formula, X in2 X out2 These are the input and output features of the detail enhancement module, respectively, with X4~X6 being intermediate features; Conv is the convolution operation, BN is the batch normalization layer, and ReLU is the activation function.

3. The rice yield prediction method based on dual-branch context fusion according to claim 1, characterized in that, The expression for the context aggregation module is as follows: X7=LayerNorm(X in3 ), X8 = SiLU(Linear(X7)), X9=LayerNorm(SS2D(SiLU(DWConv 3×3 (Linear(X7))))), X out3 =Linear(X8 X9), In the formula, X in3 X out3 X7~X9 are the input and output features of the context aggregation module, respectively; Linear is the linear layer; SiLU is the activation function; DWConv is the depthwise separable convolution; SS2D is the two-dimensional selective scanning unit. This is element-wise multiplication.

4. The rice yield prediction method based on dual-branch context fusion according to claim 3, characterized in that, The two-dimensional selective scanning unit processes the input features according to the following steps: The input features of the two-dimensional selective scanning unit are scanned and expanded along four different preset scanning directions to obtain four one-dimensional feature sequences; For each of the one-dimensional feature sequences, a selective state-space model that depends on the input is used to model the sequence, resulting in a context-enhanced one-dimensional output sequence. The one-dimensional output sequence is scanned and merged in the opposite direction to the preset scanning direction to restore a feature map with the same input feature dimension as the two-dimensional selective scanning unit. The four feature maps are fused to obtain the output features of the two-dimensional selective scanning unit.

5. The rice yield prediction method based on dual-branch context fusion according to claim 4, characterized in that, For each of the one-dimensional feature sequences, a selective state-space model that depends on the input is used for sequence modeling to obtain a context-enhanced one-dimensional output sequence, including: Based on the one-dimensional feature sequence, a time step is generated. Parameters B and C; According to the time step The parameters B and the preset parameter A are used to obtain the parameters for recursive calculation. and parameters ; Based on the parameters ,parameter The one-dimensional feature sequence is recursively calculated using parameter C and preset parameter D to generate the one-dimensional output sequence.

6. The rice yield prediction method based on dual-branch context fusion according to claim 1, characterized in that, The feature fusion module processes the input features according to the following steps: The output features of the detail enhancement module and the context aggregation module are subjected to adaptive weight vector generation processing to obtain the first weight factor and the second weight factor corresponding to the two output features, respectively. The output features of the detail enhancement module are weighted using the first weighting factor, and the output features of the context aggregation module are weighted using the second weighting factor. The two weighted features are then concatenated by channel to obtain the first fused feature. The first fused feature is sequentially convolved and activated by GeLU to generate a gated signal; The first fused feature is sequentially subjected to pointwise convolution, 3×3 depthwise separable convolution, and 5×5 depthwise separable convolution to obtain multi-scale spatial features. The second fused feature is obtained by performing element-wise multiplication on the gated signal and the multi-scale spatial features. The second fused feature is sequentially processed through pointwise convolution and layer normalization to obtain the output feature of the feature fusion module.

7. The rice yield prediction method based on dual-branch context fusion according to claim 6, characterized in that, The output features of the detail enhancement module and the context aggregation module are subjected to adaptive weight vector generation processing to obtain a first weight factor and a second weight factor corresponding to the two output features, respectively, including: The output features of the detail enhancement module and the context aggregation module are concatenated along the channel dimension to obtain the third fused feature; The third fusion feature is subjected to global average pooling to obtain a fusion vector that summarizes the information of the two branches; The fusion vector is passed through a linear layer and a scaled sigmoid activation layer to generate the first weight factor and the second weight factor.

8. The rice yield prediction method based on dual-branch context fusion according to claim 7, characterized in that, The fusion vector is passed through a linear layer and a scaled sigmoid activation layer to generate the first weight factor and the second weight factor, including: The fusion vector is activated through a linear layer and a scaled Sigmoid to generate a first weight factor and a second weight factor in the range [0.8, 1.2].

9. The rice yield prediction method based on dual-branch context fusion according to claim 1, characterized in that, The number of stacked feature extraction blocks is equal to 10.

Images