A Grape Berry Segmentation Method Based on Wavelet Transform Combined with Visual State Space Model during Fruit Thinning

Abstract

A grape berry segmentation method based on wavelet transform combined with a visual state-space model, belonging to the field of image processing technology, is proposed. The method includes: constructing a dataset and performing preprocessing and enhancement; constructing a dual-branch enhancement network; a Stem module receiving the input image and performing preliminary feature extraction; the output features flowing through a three-level WT-Mamba stage module; each WT-Mamba stage generating multi-dimensional feature maps, which are then fused to obtain global features; model training and optimization; and real-time berry segmentation inference. This invention improves edge visibility under complex lighting and occlusion scenarios, enhances the representation ability of the global structure of dense berries, and improves the accuracy, robustness, and real-time inference efficiency of berry segmentation during the thinning period, providing key technical support for grape phenotypic analysis and intelligent thinning decision-making.

Description

Technical Field

[0001] This invention relates to a method for segmenting grape berries during the thinning stage based on wavelet transform combined with a visual state space model, belonging to the field of image processing technology. Background Technology

[0002] Fruit thinning is a key management practice in table grape cultivation, significantly contributing to improved fruit quality. Thinning loosens the structure of the grape bunches, preventing nutrient waste and providing ample space for fruit expansion. This allows the remaining grapes to receive more nutrients and water, resulting in more vibrant color. Thinning also improves air circulation within the bunches, reducing the risk of friction damage between fruits. Furthermore, in high-humidity environments, better ventilation effectively reduces the incidence of fungal diseases and pests, ultimately producing larger, sweeter, and more visually appealing high-quality table grapes.

[0003] Traditional fruit thinning primarily relies on manual labor. With the development of smart agriculture and computer vision technology, image recognition-based grape detection and segmentation methods have been extensively studied and applied in areas such as yield estimation, maturity monitoring, and disease identification. Grape segmentation during the thinning period provides crucial support for grape phenotypic analysis and thinning decisions. However, the segmentation accuracy during the thinning period (when berries are still developing, small in size, color close to the background, and subject to severe occlusion) remains limited, mainly due to the following problems: First, the small size, dense density, and overlapping of berries during the thinning period make it difficult for traditional convolutional networks to capture fine-grained features; second, the complex background of the grape bunches, with foliage, branch occlusion, and changes in lighting leading to unstable target features; and third, field deployment requires lightweight models with efficient inference, demanding high inference efficiency and low computational cost while maintaining accuracy. Existing complex models are difficult to directly apply to real-time field operations.

[0004] In recent years, State Space Models (SSMs) have been gradually introduced into the field of computer vision, forming Visual State Space Models (VSSMs). These models have attracted attention due to their linear time complexity and efficient modeling capabilities for long sequences, demonstrating superior global modeling performance in tasks such as crop detection and fruit recognition. However, traditional VSSMs still have the following shortcomings when dealing with orchard scenes: First, SSMs are continuous state transition structures, making them relatively weak in responding to local texture changes, edge details, and small-area targets; second, the native scanning method of VSSMs is mostly based on a single linear path, making it difficult to fully capture the structural dependencies of the ellipsoidal morphology of fruit berries during the thinning period, thus limiting the model's representation ability under complex geometric structures; third, under unfavorable conditions such as uneven lighting, similar backgrounds, and dense occlusion by fruit berries, VSSMs lack spatial sensitivity and are even less able to accurately segment small fruit berries during the thinning period.

[0005] Meanwhile, wavelet transform, as a multi-scale time-frequency analysis tool, can simultaneously characterize local details and overall contour information at different scales, possessing directional sensitivity, local stability, and sparse representation capabilities that traditional convolution and VSSM lack. Especially in the early stages of fruit growth, when individual fruits are small, texture is weak, and the difference from the background is low, wavelet decomposition can provide complementary information in both the low-frequency and high-frequency domains, helping to enhance the model's response to local abrupt changes and edge contours. However, while wavelet transform has advantages in enhancing local details, it lacks cross-regional context modeling capabilities; and VSSM, while possessing powerful global correlation modeling capabilities, is insufficient in perceiving local details.

[0006] Using any single method alone is insufficient to adequately address the characteristics of fruit segments during the thinning period, which are characterized by small size, dense occlusion, and complex morphological structures. Therefore, deeply coupling the local multi-scale feature enhancement capabilities of wavelet transform with the long-sequence global modeling capabilities of VSSM can both compensate for the shortcomings of VSSM in capturing local fine-grained features and overcome the limitation of wavelet transform in capturing long-distance structural relationships. This provides a feasible and necessary technical path to improve the accuracy and robustness of fruit segmentation during the thinning period. Summary of the Invention

[0007] To address the problems existing in the background technology, this invention provides a method for grape berry segmentation during the thinning period based on wavelet transform combined with a visual state space model.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for grape berry segmentation during the thinning stage based on wavelet transform combined with a visual state space model, the method comprising the following steps:

[0009] S1: Constructing a grape segmentation dataset:

[0010] Based on the BBCH of the grape growth cycle, image data of the thinning period from the fruit setting stage to the fruit development stage were collected in a targeted manner. Target areas in the images were annotated at the pixel level using professional annotation tools to form a complete structured dataset containing image data and annotation information.

[0011] S2: Data Preprocessing and Augmentation

[0012] After performing standardized preprocessing on the dataset, it is divided into training set, validation set and test set according to a preset ratio. During the model training phase, online data augmentation strategies are used to dynamically generate diverse samples to expand the distribution range of training data.

[0013] S3: The dual-branch augmentation network constructs and outputs global features F:

[0014] A backbone feature extraction network was constructed, consisting of a Stem module for initial feature extraction and a cascaded three-stage WT-Mamba phase module for enhancement.

[0015] The Stem module includes three convolutional layers with 7×7 kernels connected in series and one convolutional layer with 3×3 kernels.

[0016] The 7×7 convolutional kernel captures low-level spatial features, which are used to extract contextual information from the input image;

[0017] The 3×3 convolution kernel is used to perform feature refinement and spatial downsampling operations simultaneously.

[0018] The three-level WT-Mamba stage module adopts a dual-branch parallel structure of MBWT branch and MambaEncoder branch in each WT-Mamba stage.

[0019] S3 includes the following steps:

[0020] S301: The Stem module receives the input grape image and performs preliminary extraction of low-level texture and edge features, as well as spatial dimension compression, through a combination of multiple convolutional layers; the input grape image... for:

[0021]

[0022] In the formula:

[0023] This indicates the number of images input to the Stem module at one time;

[0024] This indicates the number of color channels in the input image;

[0025] These represent the height and width of the input image, respectively.

[0026] S302: The features output after processing by the Stem module flow sequentially through the three-level WT-Mamba stage module;

[0027] S303: Each level of the WT-Mamba stage generates a multi-dimensional feature map through the synergistic effect of MBWT enhancement and MambaEncoder. Finally, the feature maps output by the three levels of the WT-Mamba stage modules are fused to obtain the global feature F of the final output of the backbone.

[0028] S303 includes the following steps:

[0029] S30301: Perform Patch Embedding processing on the input features:

[0030] The two-dimensional features are divided into K×K non-overlapping image blocks by using a convolutional layer with a kernel size of K×K and a stride of K. Each image block is linearly projected and mapped to a fixed-dimensional feature token, which is then flattened into a one-dimensional sequence for processing by the state space model.

[0031] S30301 includes the following steps:

[0032] S3030101: The input image is processed by a convolutional layer with a kernel size of K×K and a stride of K. Perform a patch convolution operation to obtain the output feature map. :

[0033]

[0034] In the formula:

[0035] This represents a convolution operation with a kernel size of K×K and a stride of K;

[0036] This indicates the number of channels output by Patch Embedding;

[0037] This indicates the height of the output image after Patch processing;

[0038] This indicates the width of the output image after Patch processing;

[0039] Indicates the padding parameters of the convolutional layer;

[0040] S3030102: Feature map implement Operations, its spatial dimensions Flatten into a token sequence :

[0041]

[0042] S3030103: For token sequences Perform a layer normalization operation to obtain the final output token sequence. :

[0043]

[0044] In the formula:

[0045] This indicates a normalization operation;

[0046] Indicates the number of tokens.

[0047] S30302: MBWT Branch Enhancement:

[0048] The output token sequence is input into the MBWT branch. The features are decomposed into LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band and HH diagonal high-frequency sub-band through multi-layer bidirectional wavelet transform. After performing 1×1 convolution enhancement on each sub-band, cross-scale feature fusion is achieved by reconstructing through inverse Haar wavelet transform, highlighting the texture of fruit edge and inter-fruit gap.

[0049] S30302 includes the following steps:

[0050] S3030201: Multi-scale bidirectional Haar wavelet decomposition:

[0051] The input feature map is fed into the MBWT branch, and the input features are decomposed through multi-level bidirectional Haar wavelet transform to obtain multi-frequency scale features. ,in: This represents wavelet transform, used to achieve frequency and decomposition of features; The spatial size is half that of the original feature map, and the number of channels is... ; This represents the global channel percentage coefficient. This represents the original number of feature channels, with the coefficient 4 derived from one low-frequency component and three high-frequency components obtained from the Haar wavelet transform decomposition.

[0052] The Haar wavelet transform filters include low-pass filters. and high-pass filter bank , , ;

[0053] S3030202: Subband Local Convolution Enhancement:

[0054] Multi-frequency scale features Each sub-band undergoes a local convolution operation to extract and optimize its feature information.

[0055] S3030203: Inverse Haar wavelet reconstruction and fusion:

[0056] Perform inverse Haar wavelet transform on each sub-band feature after convolution enhancement to restore the processed frequency features to the original feature map size, thus obtaining the cross-scale fused features. ,in: This represents the inverse Haar wavelet transform, used to restore the processed frequency features to their original spatial dimensions.

[0057] low-pass filter Used to preserve low-frequency structural information of features and filter high-frequency noise;

[0058] High-pass filter banks are used to preserve high-frequency detail information of features, enabling enhanced extraction of key local details to highlight the gradient response at the boundaries between fruit and background and fruit and stem, thereby improving the visibility of fruit edges under complex lighting and leaf shading.

[0059] S30303: MambaEncoder Branch Modeling:

[0060] The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the Cross-Selective Scan module expands the features along the rows, columns, diagonals and in the reverse direction into eight sets of one-dimensional sequences, which are then input into the continuous state space model for long-distance dependency modeling.

[0061] S30303 includes the following steps:

[0062] S3030301: Linear Projection and Two-Branch Partitioning:

[0063] For the input token sequence Perform a linear projection operation to map to a higher dimension G, resulting in two branch features. Its dimensions are Branch features As input to the gated branch, the branch features As input to the SSM branch;

[0064] S3030302: Gated branch weight generation:

[0065] branch features Applying the GELU activation function yields the gate weights. ;

[0066] S3030303: Local feature enhancement of the SSM branch:

[0067] Branch features Rearranged into two-dimensional features according to spatial dimensions For two-dimensional features Perform depthwise convolution to capture local neighborhood relationships of the fruit particles, aggregate local texture and shape information, and obtain processed features. ,in: Represents a depthwise convolution function;

[0068] S3030304: Feature sequence expansion:

[0069] Two-dimensional features are obtained through the CrossScan module. Expand along the rows, columns, diagonals, and their reverse directions to form eight one-dimensional sequences. ,in: ; Representing the The one-dimensional sequence obtained by group expansion, ;

[0070] S3030305: State-space recursive modeling:

[0071] For each group of one-dimensional sequences Perform continuous state-space recursive calculations, where:

[0072] The hidden state update formula is:

[0073]

[0074] The output formula is:

[0075]

[0076] In the formula:

[0077] Indicates the first The implicit state of a road;

[0078] Indicates the first The input sequence of the path;

[0079] These represent the learnable state transition matrix, input matrix, output matrix, and bypass mapping, respectively.

[0080] S3030306: Multi-path feature fusion:

[0081] Sequence of outputs from an eight-way state space Summation and fusion are performed to obtain a two-dimensional feature map after global state space modeling. ;

[0082] S3030307: Gated Fusion:

[0083] Gating weights With SSM branch output Perform an element-wise multiplication gated fusion operation to obtain the final output of the MambaEncoder branch. ,in: This is for element-wise multiplication.

[0084] The three-level WT-Mamba stage module described in S303 outputs feature maps with different spatial resolutions and semantic levels, specifically:

[0085] Feature map C2: Output from the first WT-Mamba stage, with the highest spatial resolution, containing details of the grain texture;

[0086] Feature map C3: Output from the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution;

[0087] Feature map C5: Output from the third WT-Mamba stage, containing the highest-level granule semantic structure information.

[0088] S30304: Dual-branch fusion:

[0089] Features output by the MBWT branch Features of the output of the MambaEncoder branch Aligned to the same spatial dimension, the dual-branch feature fusion is achieved through pixel-by-pixel weighted summation, resulting in the final output features of the three-level WT-Mamba stage module.

[0090] S4: Model Training and Optimization

[0091] The global feature F is input into the segmentation detection head for training. The model parameters are optimized by minimizing the difference between the prediction results and the true labels. During the training process, the accuracy of the validation set and the segmentation IoU value are used as indicators to iteratively select the model parameters with the best generalization ability.

[0092] S5: Real-time Particle Segmentation Inference:

[0093] The grape thinning images to be segmented are input into the trained optimal model. After feature extraction and bi-branch enhancement, the output includes the grape outline, confidence level, and defect markers.

[0094] Compared with the prior art, the beneficial effects of the present invention are:

[0095] This invention utilizes a multi-layer bidirectional wavelet transform module to deconstruct and reconstruct features in the low-frequency to high-frequency domain, enhancing the gradient response of local details such as fruit edges and inter-fruit gaps, and improving edge visibility under complex lighting and occlusion scenarios. Through multi-path scanning and state-space recursion mechanisms of the visual state-space model, it captures the structural dependencies and long-distance spatial associations of the ellipsoidal morphology of the fruit bunch, enhancing the representation of the global structure of dense fruit. A dual-branch feature fusion mechanism achieves complementarity between spatial and frequency domain information, compensating for the shortcomings of traditional convolutional networks in capturing fine-grained features and overcoming the limitations of wavelet transform in lacking contextual modeling capabilities. Ultimately, while ensuring a lightweight model, it significantly improves the accuracy, robustness, and real-time inference efficiency of fruit segmentation during thinning, providing key technical support for grape phenotypic analysis and intelligent thinning decisions. Attached Figure Description

[0096] Figure 1 This is a schematic diagram of the wavelet transform combined with the visual state space model network structure of the present invention;

[0097] Figure 2 This is a schematic diagram illustrating the segmentation effect of the grape berry segmentation model and the comparative experimental model of the present invention. Detailed Implementation

[0098] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0099] A method for grape berry segmentation during thinning based on wavelet transform combined with a visual state space model, the method comprising the following steps:

[0100] S1: Constructing a grape segmentation dataset:

[0101] Based on the BBCH (Biologische Bundesanstalt, Bundessortenamt und Chemische Industrie, a unified coding system for plant growth cycles) of grapes, image data of key nodes from fruit setting to fruit thinning were collected. Targeted annotation of the target regions (grape berry region, berry surface defects, and bunch background) in the images was performed at the pixel level using professional annotation tools, forming a complete structured dataset containing image data and annotation information. This provides a high-quality data foundation for subsequent algorithm training and performance evaluation.

[0102] S2: Data Preprocessing and Augmentation

[0103] After performing standardized preprocessing on the dataset, it is divided into training set, validation set and test set according to a preset ratio. During the model training stage, online data augmentation strategy is used to dynamically generate diversified samples, expand the distribution range of training data, and improve the robustness and generalization ability of the model under different working conditions and various defect forms.

[0104] The online data augmentation strategy includes one or more combinations of random rotation, scaling, mirroring, and cropping.

[0105] S3: The dual-branch augmentation network constructs and outputs global features F:

[0106] A backbone feature extraction network was constructed, consisting of a Stem module for initial feature extraction and a cascaded three-stage WT-Mamba phase module for enhancement.

[0107] For the semantic segmentation task of grape berry images during the thinning stage, a three-level WT-Mamba stage module is used to replace the single-path convolution or self-attention structure in the traditional coding network.

[0108] The three-level WT-Mamba stage module can gradually expand the receptive field, strengthen the contextual relationship, and capture the target morphological differences in grape growth stages at different semantic levels, thereby providing highly discriminative feature support for subsequent segmentation heads.

[0109] S4: Model Training and Optimization

[0110] The global feature F is input into the segmentation detection head for training. The model parameters are optimized by minimizing the difference between the predicted results and the ground truth annotations, allowing the model to gradually learn the discrimination features of grape targets under different scales and occlusions. During training, the validation set accuracy and segmentation IoU are used as indicators to iteratively select the model parameters with the best generalization ability.

[0111] S5: Real-time Particle Segmentation Inference:

[0112] The grape thinning images to be segmented are input into the trained optimal model. After feature extraction and bi-branch enhancement, the output includes the grape outline, confidence level, and defect markers.

[0113] Furthermore, the Stem module described in S3 adopts a simple architecture design, including three convolutional layers with 7×7 convolutional kernels connected in series and one convolutional layer with 3×3 convolutional kernels.

[0114] The 7×7 convolutional kernel, with its relatively large receptive field, can effectively capture low-level spatial features and is used to extract rich contextual information from the input image.

[0115] The 3×3 convolution kernel is used to perform feature refinement and spatial downsampling operations simultaneously.

[0116] This process reduces the resolution of the feature map, preserving key information while reducing computational complexity in subsequent stages.

[0117] In complex orchard environments, the large receptive field employed by the Stem module can effectively capture contextual information surrounding the grape berry structure, thereby efficiently extracting shallow texture features.

[0118] Furthermore, each level of the three-level WT-Mamba stage module described in S3 adopts a dual-branch parallel structure and fusion mechanism of MBWT (multi-scale bidirectional wavelet enhancement) branch and MambaEncoder (visual state space modeling) branch, which enhances the texture of fruit grain edges and long-distance spatial correlation while keeping the computational load controllable.

[0119] The MambaEncoder branch excels at modeling global structural relationships, while the MBWT branch excels at extracting local textures and edge details. During the fusion stage, the two are stacked element-wise, enabling the model to simultaneously utilize feature representations of different frequencies, scales, and structural dimensions. This achieves complementary enhancement of local fine textures and global topological information, significantly improving the detection and segmentation of densely packed small targets such as grape berries.

[0120] By merging the MambaEncoder branch and the MBWT branch in parallel, we have achieved efficient modeling of subtle edges, textures and long-range dependencies between the grape berry region and the vine background, which has significant structural innovation and application advantages.

[0121] Furthermore, step S3 includes the following steps:

[0122] S301: The Stem module receives a standardized grape image as input and performs preliminary extraction of low-level texture and edge features, as well as spatial dimension compression, through a combination of multiple convolutional layers; the input grape image... for:

[0123]

[0124] In the formula:

[0125] This indicates the batch size, which is the number of images input to the Stem module at one time.

[0126] This indicates the number of color channels in the input image;

[0127] These represent the height and width of the input image, respectively.

[0128] S302: The features output after processing by the Stem module flow sequentially through the three-level WT-Mamba stage module;

[0129] S303: Each level of the WT-Mamba stage generates multi-dimensional feature maps through the synergistic effect of MBWT enhancement and MambaEncoder, realizing deep semantic representation of grape berries, inflorescence structure and background environment. Finally, the feature maps output by the three-level WT-Mamba stage modules are fused to obtain the global feature F of the final output of the main body.

[0130] Furthermore, step S303 includes the following steps:

[0131] S30301: Perform Patch Embedding processing on the input features:

[0132] The two-dimensional features are divided into K×K non-overlapping image blocks by using a convolutional layer with a kernel size of K×K and a stride of K. Each image block is linearly projected and mapped to a fixed-dimensional feature token, which is then flattened into a one-dimensional sequence for processing by the state space model.

[0133] S30302: MBWT Branch Enhancement:

[0134] The output token sequence is input into the MBWT branch. The features are decomposed into LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band and HH diagonal high-frequency sub-band through multi-layer bidirectional wavelet transform. After performing 1×1 convolution enhancement on each sub-band, cross-scale feature fusion is achieved by reconstructing through inverse Haar wavelet transform, highlighting the texture of the fruit edge and the gap between the fruit, which is helpful to distinguish the tightly clustered small fruit in complex background.

[0135] S30303: MambaEncoder Branch Modeling:

[0136] The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the Cross-Selective Scan module expands the features into eight sets of one-dimensional sequences along the rows, columns, diagonals and in the reverse direction. These sequences are then input into the continuous state space model for long-distance dependency modeling, capturing the structural relationship of the entire bunch of grapes and branches, and avoiding the fragmentation caused by focusing only on local areas.

[0137] S30304: Dual-branch fusion:

[0138] Features output by the MBWT branch Features of the output of the MambaEncoder branch Aligned to the same spatial dimension, the dual-branch feature fusion is achieved through pixel-by-pixel weighted summation, resulting in the final output features of the three-level WT-Mamba stage module.

[0139] This invention introduces the Visual Mamba State Space Model (VSSM) into feature encoding, constructing a dynamic matrix recursive network capable of performing multi-directional scan modeling in two-dimensional space. By generating eight scan sequences through Cross-SelectiveScan, the model can simultaneously capture spatial dependencies in the horizontal, vertical, and diagonal directions. Furthermore, it utilizes a learnable state transition matrix A, input / output mapping matrices B and C, and a fixed dimension to establish a dynamic modeling capability for long-distance structural dependencies.

[0140] This continuous modeling method based on state-space equations has the advantages of low computational cost and high scalability, and is particularly suitable for capturing key information such as the overall shape, arrangement direction, and occlusion relationship of grape berries in high-resolution grape images.

[0141] Furthermore, step S30301 includes the following steps:

[0142] S3030101: The input image is processed by a convolutional layer with a kernel size of K×K and a stride of K. Perform a patch convolution operation to obtain the output feature map. :

[0143]

[0144] In the formula:

[0145] This represents a convolution operation with a kernel size of K×K and a stride of K;

[0146] This indicates the number of channels output by Patch Embedding;

[0147] This indicates the height of the output image after Patch processing;

[0148] This indicates the width of the output image after Patch processing;

[0149] Indicates the padding parameters of the convolutional layer;

[0150] S3030102: Feature map implement Operations, its spatial dimensions Flatten into a token sequence :

[0151]

[0152] S3030103: Treat each spatial location as a token sequence, and for the token sequence... Perform a layer normalization operation to obtain the final output token sequence. :

[0153]

[0154] In the formula:

[0155] This indicates a normalization operation;

[0156] Indicates the number of tokens.

[0157] Furthermore, step S30302 includes the following steps:

[0158] S3030201: Multi-scale bidirectional Haar wavelet decomposition:

[0159] The input feature map is fed into the MBWT branch, and the input features are decomposed through multi-level bidirectional Haar wavelet transform (WT) to obtain multi-frequency scale features. ,in: This represents wavelet transform, used to achieve frequency and decomposition of features; The spatial size is half that of the original feature map, and the number of channels is... ; This represents the global channel percentage coefficient. This represents the original number of feature channels, with the coefficient 4 derived from one low-frequency component and three high-frequency components obtained from the Haar wavelet transform decomposition.

[0160] The Haar wavelet transform filters include low-pass filters. and high-pass filter bank , , ;

[0161] S3030202: Subband Local Convolution Enhancement:

[0162] Multi-frequency scale features Local convolution operations (Conv) are performed on each subband (low-frequency component LL, horizontal high-frequency component LH, vertical high-frequency component HL, and diagonal high-frequency component HH) to extract and optimize the feature information of each subband.

[0163] S3030203: Inverse Haar wavelet reconstruction and fusion:

[0164] Perform inverse Haar wavelet transform (IWT) on each sub-band feature after convolution enhancement to restore the processed frequency features to the original feature map size, thus obtaining the cross-scale fused features. ,in: This represents the inverse Haar wavelet transform, used to restore the processed frequency features to their original spatial dimensions.

[0165] low-pass filter Low-frequency structural information (corresponding to smooth areas of the image, reflecting the overall trend) is used to preserve features, while high-frequency noise is filtered out.

[0166] The high-pass filter bank is used to preserve high-frequency detail information of features (corresponding to the edges and textures of the image) to enhance the extraction of key local details. The wavelet branch explicitly separates low-frequency structure and high-frequency details at multiple scales to highlight the gradient response at the boundaries between fruit and background and between fruit and stem, improve the visibility of fruit edges under complex lighting and leaf occlusion, and help the subsequent decoder obtain more accurate boundary information.

[0167] Furthermore, step S30303 includes the following steps:

[0168] S3030301: Linear Projection and Two-Branch Partitioning:

[0169] For the input token sequence Perform a linear projection operation to map to a higher dimension G, resulting in two branch features. Its dimensions are Branch features As input to the gated branch, the branch features As input to the SSM (state space) branch;

[0170] S3030302: Gated branch weight generation:

[0171] branch features Applying the GELU activation function yields the gate weights. The subsequent SSM output is dynamically weighted to enhance the response to key fruit grain regions;

[0172] S3030303: Local feature enhancement of the SSM branch:

[0173] Branch features Rearranged into two-dimensional features according to spatial dimensions For two-dimensional features Perform depthwise convolution to capture local neighborhood relationships of the fruit particles, aggregate local texture and shape information, and obtain processed features. ,in: This represents a depthwise convolution function; it effectively enhances the local consistency of the grain region and extracts local patterns such as texture, spots, and bumps.

[0174] S3030304: Feature sequence expansion:

[0175] Two-dimensional features are obtained through the CrossScan module. Expand along the rows, columns, diagonals, and their reverse directions to form eight one-dimensional sequences. ,in: ; Representing the The one-dimensional sequence obtained by group expansion, ;

[0176] S3030305: State-space recursive modeling:

[0177] For each group of one-dimensional sequences Perform continuous state-space recursive calculations, where:

[0178] The hidden state update formula is:

[0179]

[0180] The output formula is:

[0181]

[0182] In the formula:

[0183] Indicates the first The implicit state of a road;

[0184] Indicates the first The input sequence of the path;

[0185] These represent the learnable state transition matrix, input matrix, output matrix, and bypass mapping, respectively.

[0186] This structure enables the model to capture long-range dependencies in any direction and maintain global consistency even when the granules are dense and occluded.

[0187] S3030306: Multi-path feature fusion:

[0188] Sequence of outputs from an eight-way state space Summation and fusion are performed to obtain a two-dimensional feature map after global state space modeling. ;

[0189] S3030307: Gated Fusion:

[0190] Gating weights With SSM branch output Perform an element-wise multiplication gated fusion operation to obtain the final output of the MambaEncoder branch. ,in: This is for element-wise multiplication (one-to-one correspondence between the same position and the same channel).

[0191] In the MambaEncoder branch, DWConv is used to perform detailed texture enhancement on each channel within the local grain region, strengthening local structures such as grain edges and gaps between grains. CrossScan stretches the entire image into a sequence along the horizontal, vertical, and diagonal lines, allowing the state-space model to capture long-range dependencies in these directions. Multi-path SSM output overlay allows the model to simultaneously perceive horizontal, vertical, and diagonal structural relationships, improving its ability to model overlapping grains. In the nonlinear gating branch, the GELU gating mechanism is used to amplify the long-range dependency information of the SSM by using a large gating value at grain edges and overlapping areas, while lowering the gating value in areas with interference such as pure background and light spots to prevent the SSM from propagating noise further.

[0192] The features obtained by using the MambaEncoder branch have both multi-directional long-range dependencies (which are beneficial for identifying the connectivity structure of the whole bunch of grapes) and local departmental noise suppression (which reduces interference from background elements such as leaves and branches), which significantly helps to improve the integrity of the grape berry segmentation outline and the separation of clumps of grapes during the thinning period.

[0193] The MBWT branch introduced in the WT-Mamba stage can perform multi-scale, bidirectional decomposition of input features using orthogonal wavelet bases before the feature map enters deep modeling. This decomposition process captures local high-frequency features of grape berry edges, highlights, bloom textures, and gaps between berries at different spatial frequencies, and achieves learnable frequency-selective enhancement through sub-band convolution, thereby strengthening the structural information of berry boundaries and occluded areas and improving the ability of subsequent segmentation models to distinguish dense berries.

[0194] Furthermore, the three-level WT-Mamba stage module described in S303 outputs feature maps with different spatial resolutions and semantic levels, specifically:

[0195] Feature map C2: Output from the first WT-Mamba stage, with the highest spatial resolution and rich details of grain texture;

[0196] Feature map C3: Output from the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution;

[0197] Feature map C5: Output from the third WT-Mamba stage, containing the highest-level granule semantic structure information.

[0198] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0199] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for grape berry segmentation during thinning based on wavelet transform combined with a visual state-space model, characterized in that: The method includes the following steps: S1: Constructing a grape segmentation dataset: Based on the BBCH of the grape growth cycle, image data of the thinning period from the fruit setting stage to the fruit development stage were collected in a targeted manner. Target areas in the images were annotated at the pixel level using professional annotation tools to form a complete structured dataset containing image data and annotation information. S2: Data Preprocessing and Augmentation After performing standardized preprocessing on the dataset, it is divided into training set, validation set and test set according to a preset ratio. During the model training phase, online data augmentation strategies are used to dynamically generate diverse samples to expand the distribution range of training data. S3: The dual-branch augmentation network constructs and outputs global features F: A backbone feature extraction network was constructed, consisting of a Stem module for initial feature extraction and a cascaded three-stage WT-Mamba phase module for enhancement. S3 includes the following steps: S301: The Stem module receives the input grape image and performs preliminary extraction of low-level texture and edge features, as well as spatial dimension compression, through a combination of multiple convolutional layers; the input grape image... for: In the formula: This indicates the number of images input to the Stem module at one time; This indicates the number of color channels in the input image; These represent the height and width of the input image, respectively. S302: The features output after processing by the Stem module flow sequentially through the three-level WT-Mamba stage module; S303: Each level of the WT-Mamba stage generates a multi-dimensional feature map through the synergistic effect of MBWT enhancement and MambaEncoder. Finally, the feature maps output by the three levels of the WT-Mamba stage modules are fused to obtain the global feature F of the final output of the backbone. S303 includes the following steps: S30301: Perform Patch Embedding processing on the input features: The two-dimensional features are divided into K×K non-overlapping image blocks by using a convolutional layer with a kernel size of K×K and a stride of K. Each image block is linearly projected and mapped to a fixed-dimensional feature token, which is then flattened into a one-dimensional sequence for processing by the state space model. S30302: MBWT Branch Enhancement: The output token sequence is input into the MBWT branch. The features are decomposed into LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band and HH diagonal high-frequency sub-band through multi-layer bidirectional wavelet transform. After performing 1×1 convolution enhancement on each sub-band, cross-scale feature fusion is achieved by reconstructing through inverse Haar wavelet transform, highlighting the texture of fruit edge and inter-fruit gap. S30303: MambaEncoder Branch Modeling: The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the Cross-Selective Scan module expands the features along the rows, columns, diagonals and in the reverse direction into eight sets of one-dimensional sequences, which are then input into the continuous state space model for long-distance dependency modeling. S30304: Dual-branch fusion: Features output by the MBWT branch Features of the output of the MambaEncoder branch Aligned to the same spatial dimension, dual-branch feature fusion is achieved by weighted summation of pixels to obtain the final output features of the three-level WT-Mamba stage module; S4: Model Training and Optimization The global feature F is input into the segmentation detection head for training. The model parameters are optimized by minimizing the difference between the prediction results and the true labels. During the training process, the accuracy of the validation set and the segmentation IoU value are used as indicators to iteratively select the model parameters with the best generalization ability. S5: Real-time Particle Segmentation Inference: The grape thinning images to be segmented are input into the trained optimal model. After feature extraction and bi-branch enhancement, the output includes the grape outline, confidence level, and defect markers.

2. The grape berry segmentation method based on wavelet transform combined with a visual state space model according to claim 1, characterized in that: The Stem module described in S3 includes three convolutional layers with 7×7 kernels connected in series and one convolutional layer with 3×3 kernels. The 7×7 convolutional kernel captures low-level spatial features, which are used to extract contextual information from the input image; The 3×3 convolution kernel is used to perform feature refinement and spatial downsampling operations simultaneously.

3. The grape berry segmentation method based on wavelet transform combined with a visual state space model according to claim 2, characterized in that: The three-level WT-Mamba stage module described in S3 adopts a dual-branch parallel structure of MBWT branch and MambaEncoder branch at each level of the WT-Mamba stage module.

4. The grape berry segmentation method based on wavelet transform combined with a visual state space model according to claim 3, characterized in that: S30301 includes the following steps: S3030101: The input image is processed by a convolutional layer with a kernel size of K×K and a stride of K. Perform a patch convolution operation to obtain the output feature map. : In the formula: This represents a convolution operation with a kernel size of K×K and a stride of K; This indicates the number of channels output by Patch Embedding; This indicates the height of the output image after Patch processing; This indicates the width of the output image after Patch processing; Indicates the padding parameters of the convolutional layer; S3030102: Feature map implement Operations, its spatial dimensions Flatten into a token sequence : S3030103: For token sequences Perform a layer normalization operation to obtain the final output token sequence. : In the formula: This indicates a normalization operation; Indicates the number of tokens.

5. The grape berry segmentation method based on wavelet transform combined with a visual state space model according to claim 4, characterized in that: S30302 includes the following steps: S3030201: Multi-scale bidirectional Haar wavelet decomposition: The input feature map is fed into the MBWT branch, and the input features are decomposed through multi-level bidirectional Haar wavelet transform to obtain multi-frequency scale features. ,in: This represents wavelet transform, used to achieve frequency and decomposition of features; The spatial size is half that of the original feature map, and the number of channels is... ; This represents the global channel percentage coefficient. This represents the original number of feature channels, with the coefficient 4 derived from one low-frequency component and three high-frequency components obtained from the Haar wavelet transform decomposition. The Haar wavelet transform filters include low-pass filters. and high-pass filter bank , , ; S3030202: Subband Local Convolution Enhancement: Multi-frequency scale features Each sub-band undergoes a local convolution operation to extract and optimize its feature information. S3030203: Inverse Haar wavelet reconstruction and fusion: Perform inverse Haar wavelet transform on each sub-band feature after convolution enhancement to restore the processed frequency features to the original feature map size, thus obtaining the cross-scale fused features. ,in: This represents the inverse Haar wavelet transform, used to restore the processed frequency features to their original spatial dimensions. low-pass filter Used to preserve low-frequency structural information of features and filter high-frequency noise; High-pass filter banks are used to preserve high-frequency detail information of features, enabling enhanced extraction of key local details to highlight the gradient response at the boundaries between fruit and background and fruit and stem, thereby improving the visibility of fruit edges under complex lighting and leaf shading.

6. The grape berry segmentation method based on wavelet transform combined with a visual state space model according to claim 5, characterized in that: S30303 includes the following steps: S3030301: Linear Projection and Two-Branch Partitioning: For the input token sequence Perform a linear projection operation to map to a higher dimension G, resulting in two branch features. Its dimensions are Branch features As input to the gated branch, the branch features As input to the SSM branch; S3030302: Gated branch weight generation: branch features Applying the GELU activation function yields the gate weights. ; S3030303: Local feature enhancement of the SSM branch: Branch features Rearranged into two-dimensional features according to spatial dimensions For two-dimensional features Perform depthwise convolution to capture local neighborhood relationships of the fruit particles, aggregate local texture and shape information, and obtain processed features. ,in: Represents a depthwise convolution function; S3030304: Feature sequence expansion: Two-dimensional features are obtained through the CrossScan module. Expand along the rows, columns, diagonals, and their reverse directions to form eight one-dimensional sequences. ,in: ; Representing the The one-dimensional sequence obtained by group expansion, ; S3030305: State-space recursive modeling: For each group of one-dimensional sequences Perform continuous state-space recursive calculations, where: The hidden state update formula is: The output formula is: In the formula: Indicates the first The implicit state of a road; Indicates the first The input sequence of the path; These represent the learnable state transition matrix, input matrix, output matrix, and bypass mapping, respectively. S3030306: Multi-path feature fusion: Sequence of outputs from an eight-way state space Summation and fusion are performed to obtain a two-dimensional feature map after global state space modeling. ; S3030307: Gated Fusion: Gating weights With SSM branch output Perform an element-wise multiplication gated fusion operation to obtain the final output of the MambaEncoder branch. ,in: This is for element-wise multiplication.

7. A method for grape berry segmentation during thinning based on wavelet transform combined with a visual state-space model, as described in claim 4 or 6, characterized in that: The three-level WT-Mamba stage module described in S303 outputs feature maps with different spatial resolutions and semantic levels, specifically: Feature map C2: Output from the first WT-Mamba stage, with the highest spatial resolution, containing details of the grain texture; Feature map C3: Output from the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution; Feature map C5: Output from the third WT-Mamba stage, containing the highest-level granule semantic structure information.

Images