Grape fruit thinning period fruit grain segmentation method based on combination of wavelet transform and visual state space model

Abstract

The invention discloses a grape fruit grain segmentation method in a fruit thinning period based on combination of wavelet transform and a visual state space model, and belongs to the technical field of image processing. The method comprises the steps that a data set is constructed, preprocessed and enhanced; constructing a double-branch enhanced network; the Stem module receives an input image to complete feature preliminary extraction; the output characteristic flows through a three-stage WT-Mama stage module; generating a multi-dimensional feature map at each stage of WT-Mama stage, and fusing to obtain a global feature; training and optimizing the model; and performing real-time fruit grain segmentation reasoning. According to the method, the edge visibility in a complex illumination and shielding scene is improved, the characterization capability of a global structure of dense fruit grains is enhanced, the precision and robustness of fruit grain segmentation in the fruit thinning period and the field real-time reasoning efficiency are improved, and a key technical support is provided for grape phenotype analysis and intelligent fruit thinning decision.

Description

Technical Field

[0001] This invention relates to a method for segmenting grape berries during the thinning stage based on wavelet transform combined with a visual state space model, belonging to the field of image processing technology. Background Technology

[0002] Fruit thinning is a key management practice in table grape cultivation, significantly contributing to improved fruit quality. Thinning loosens the structure of the grape bunches, preventing nutrient waste and providing ample space for fruit expansion. This allows the remaining grapes to receive more nutrients and water, resulting in more vibrant color. Thinning also improves air circulation within the bunches, reducing the risk of friction damage between fruits. Furthermore, in high-humidity environments, better ventilation effectively reduces the incidence of fungal diseases and pests, ultimately producing larger, sweeter, and more visually appealing high-quality table grapes.

[0003] Traditional fruit thinning primarily relies on manual labor. With the development of smart agriculture and computer vision technology, image recognition-based grape detection and segmentation methods have been extensively studied and applied in areas such as yield estimation, maturity monitoring, and disease identification. Grape segmentation during the thinning period provides crucial support for grape phenotypic analysis and thinning decisions. However, the segmentation accuracy during the thinning period (when berries are still developing, small in size, color close to the background, and subject to severe occlusion) remains limited, mainly due to the following problems: First, the small size, dense density, and overlapping of berries during the thinning period make it difficult for traditional convolutional networks to capture fine-grained features; second, the complex background of the grape bunches, with foliage, branch occlusion, and changes in lighting leading to unstable target features; and third, field deployment requires lightweight models with efficient inference, demanding high inference efficiency and low computational cost while maintaining accuracy. Existing complex models are difficult to directly apply to real-time field operations.

[0004] In recent years, State Space Models (SSMs) have been gradually introduced into the field of computer vision, forming Visual State Space Models (VSSMs). These models have attracted attention due to their linear time complexity and efficient modeling capabilities for long sequences, demonstrating superior global modeling performance in tasks such as crop detection and fruit recognition. However, traditional VSSMs still have the following shortcomings when dealing with orchard scenes: First, SSMs are continuous state transition structures, making them relatively weak in responding to local texture changes, edge details, and small-area targets; second, the native scanning method of VSSMs is mostly based on a single linear path, making it difficult to fully capture the structural dependencies of the ellipsoidal morphology of fruit berries during the thinning period, thus limiting the model's representation ability under complex geometric structures; third, under unfavorable conditions such as uneven lighting, similar backgrounds, and dense occlusion by fruit berries, VSSMs lack spatial sensitivity and are even less able to accurately segment small fruit berries during the thinning period.

[0005] Meanwhile, wavelet transform, as a multi-scale time-frequency analysis tool, can simultaneously characterize local details and overall contour information at different scales, possessing directional sensitivity, local stability, and sparse representation capabilities that traditional convolution and VSSM lack. Especially in the early stages of fruit growth, when individual fruits are small, texture is weak, and the difference from the background is low, wavelet decomposition can provide complementary information in both the low-frequency and high-frequency domains, helping to enhance the model's response to local abrupt changes and edge contours. However, while wavelet transform has advantages in enhancing local details, it lacks cross-regional context modeling capabilities; and VSSM, while possessing powerful global correlation modeling capabilities, is insufficient in perceiving local details.

[0006] Using any single method alone is insufficient to adequately address the characteristics of fruit segments during the thinning period, which are characterized by small size, dense occlusion, and complex morphological structures. Therefore, deeply coupling the local multi-scale feature enhancement capabilities of wavelet transform with the long-sequence global modeling capabilities of VSSM can both compensate for the shortcomings of VSSM in capturing local fine-grained features and overcome the limitation of wavelet transform in capturing long-distance structural relationships. This provides a feasible and necessary technical path to improve the accuracy and robustness of fruit segmentation during the thinning period. Summary of the Invention

[0007] To address the problems existing in the background technology, this invention provides a method for grape berry segmentation during the thinning period based on wavelet transform combined with a visual state space model.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for grape berry segmentation during the thinning stage based on wavelet transform combined with a visual state space model, the method comprising the following steps:

[0009] S1: Constructing a grape berry segmentation dataset:

[0010] Based on the BBCH of the grape growth cycle, image data of the thinning period from the fruit setting stage to the fruit development stage were collected in a targeted manner. Target areas in the images were annotated at the pixel level using professional annotation tools to form a complete structured dataset containing image data and annotation information.

[0011] S2: Data Preprocessing and Augmentation

[0012] After performing standardized preprocessing on the dataset, it is divided into training set, validation set and test set according to a preset ratio. During the model training phase, online data augmentation strategies are used to dynamically generate diverse samples to expand the distribution range of training data.

[0013] S3: The dual-branch augmentation network constructs and outputs global features F:

[0014] A backbone feature extraction network was constructed, consisting of a Stem module for initial feature extraction and a cascaded three-stage WT-Mamba phase module for enhancement.

[0015] The Stem module includes three convolutional layers with 7×7 kernels connected in series and one convolutional layer with 3×3 kernels.

[0016] The 7×7 convolutional kernel captures low-level spatial features, which are used to extract contextual information from the input image;

[0017] The 3×3 convolution kernel is used to perform feature refinement and spatial downsampling operations simultaneously.

[0018] The three-level WT-Mamba stage module adopts a dual-branch parallel structure of MBWT branch and MambaEncoder branch in each WT-Mamba stage.

[0019] S3 includes the following steps:

[0020] S301: The Stem module receives the input grape image and performs preliminary extraction of low-level texture and edge features, as well as spatial dimension compression, through a combination of multiple convolutional layers; the input grape image... for:

[0021]

[0022] In the formula:

[0023] This indicates the number of images input to the Stem module at one time;

[0024] This indicates the number of color channels in the input image;

[0025] These represent the height and width of the input image, respectively.

[0026] S302: The features output after processing by the Stem module flow sequentially through the three-level WT-Mamba stage module;

[0027] S303: Each level of the WT-Mamba stage generates a multi-dimensional feature map through the synergistic effect of MBWT enhancement and MambaEncoder. Finally, the feature maps output by the three levels of the WT-Mamba stage modules are fused to obtain the global feature F of the final output of the backbone.

[0028] S303 includes the following steps:

[0029] S30301: Perform Patch Embedding processing on the input features:

[0030] The two-dimensional features are divided into K×K non-overlapping image blocks by using a convolutional layer with a kernel size of K×K and a stride of K. Each image block is linearly projected and mapped to a fixed-dimensional feature token, which is then flattened into a one-dimensional sequence for processing by the state space model.

[0031] S30301 includes the following steps:

[0032] S3030101: The input image is processed by a convolutional layer with a kernel size of K×K and a stride of K. Perform a patch convolution operation to obtain the output feature map. :

[0033]

[0034] In the formula:

[0035] This represents a convolution operation with a kernel size of K×K and a stride of K;

[0036] This indicates the number of channels output by Patch Embedding;

[0037] This indicates the height of the output image after Patch processing;

[0038] This indicates the width of the output image after Patch processing;

[0039] Indicates the padding parameters of the convolutional layer;

[0040] S3030102: Feature map implement Operations, its spatial dimensions Flatten into a token sequence :

[0041]

[0042] S3030103: For token sequences Perform a layer normalization operation to obtain the final output token sequence. :

[0043]

[0044] In the formula:

[0045] This indicates a normalization operation;

[0046] Indicates the number of tokens.

[0047] S30302: MBWT Branch Enhancement:

[0048] The output token sequence is input into the MBWT branch. The features are decomposed into LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band and HH diagonal high-frequency sub-band through multi-layer bidirectional wavelet transform. After performing 1×1 convolution enhancement on each sub-band, cross-scale feature fusion is achieved by reconstructing through inverse Haar wavelet transform, highlighting the texture of fruit edge and inter-fruit gap.

[0049] S30302 includes the following steps:

[0050] S3030201: Multi-scale bidirectional Haar wavelet decomposition:

[0051] The input feature map is fed into the MBWT branch, and the input features are decomposed through multi-level bidirectional Haar wavelet transform to obtain multi-frequency scale features. ,in: This represents wavelet transform, used to achieve frequency and decomposition of features; The spatial size is half that of the original feature map, and the number of channels is... ; This represents the global channel percentage coefficient. This represents the original number of feature channels, with the coefficient 4 derived from one low-frequency component and three high-frequency components obtained from the Haar wavelet transform decomposition.

[0052] The Haar wavelet transform filters include low-pass filters. and high-pass filter bank , , ;

[0053] S3030202: Subband Local Convolution Enhancement:

[0054] Multi-frequency scale features Each sub-band undergoes a local convolution operation to extract and optimize its feature information.

[0055] S3030203: Inverse Haar wavelet reconstruction and fusion:

[0056] Perform inverse Haar wavelet transform on each sub-band feature after convolution enhancement to restore the processed frequency features to the original feature map size, thus obtaining the cross-scale fused features. ,in: This represents the inverse Haar wavelet transform, used to restore the processed frequency features to their original spatial dimensions.

[0057] low-pass filter Used to preserve low-frequency structural information of features and filter high-frequency noise;

[0058] High-pass filter banks are used to preserve high-frequency detail information of features, enabling enhanced extraction of key local details to highlight the gradient response at the boundaries between fruit and background and fruit and stem, thereby improving the visibility of fruit edges under complex lighting and leaf shading.

[0059] S30303: MambaEncoder Branch Modeling:

[0060] The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the Cross-Selective Scan module expands the features along the rows, columns, diagonals and in the reverse direction into eight sets of one-dimensional sequences, which are then input into the continuous state space model for long-distance dependency modeling.

[0061] S30303 includes the following steps:

[0062] S3030301: Linear Projection and Two-Branch Partitioning:

[0063] For the input token sequence Perform a linear projection operation to map to a higher dimension G, resulting in two branch features. Its dimensions are Branch features As input to the gated branch, the branch features As input to the SSM branch;

[0064] S3030302: Gated branch weight generation:

[0065] branch features Applying the GELU activation function yields the gate weights. ;

[0066] S3030303: Local feature enhancement of the SSM branch:

[0067] Branch features Rearranged into two-dimensional features according to spatial dimensions For two-dimensional features Perform depthwise convolution to capture local neighborhood relationships of the fruit particles, aggregate local texture and shape information, and obtain processed features. ,in: Represents a depthwise convolution function;

[0068] S3030304: Feature sequence expansion:

[0069] Two-dimensional features are processed using the CrossScan module. Expand along the rows, columns, diagonals, and their reverse directions to form eight one-dimensional sequences. ,in: ; Representing the The one-dimensional sequence obtained by group expansion, ;

[0070] S3030305: State-space recursive modeling:

[0071] For each group of one-dimensional sequences Perform continuous state-space recursive calculations, where:

[0072] The hidden state update formula is:

[0073]

[0074] The output formula is:

[0075]

[0076] In the formula:

[0077] Indicates the first The implicit state of a road;

[0078] Indicates the first The input sequence of the path;

[0079] These represent the learnable state transition matrix, input matrix, output matrix, and bypass mapping, respectively.

[0080] S3030306: Multi-path feature fusion:

[0081] Sequence of outputs from an eight-way state space Summation and fusion are performed to obtain a two-dimensional feature map after global state space modeling. ;

[0082] S3030307: Gated Fusion:

[0083] Gating weights With SSM branch output Perform an element-wise multiplication gated fusion operation to obtain the final output of the MambaEncoder branch. ,in: This is for element-wise multiplication.

[0084] The three-level WT-Mamba stage module described in S303 outputs feature maps with different spatial resolutions and semantic levels, specifically:

[0085] Feature map C2: Output from the first WT-Mamba stage, with the highest spatial resolution, containing details of the grain texture;

[0086] Feature map C3: Output from the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution;

[0087] Feature map C5: Output from the third WT-Mamba stage, containing the highest-level granule semantic structure information.

[0088] S30304: Dual-branch fusion:

[0089] Features output by the MBWT branch Features of the output of the MambaEncoder branch Aligned to the same spatial dimension, the dual-branch feature fusion is achieved through pixel-by-pixel weighted summation, resulting in the final output features of the three-level WT-Mamba stage module.

[0090] S4: Model Training and Optimization

[0091] The global feature F is input into the segmentation detection head for training. The model parameters are optimized by minimizing the difference between the prediction results and the true labels. During the training process, the accuracy of the validation set and the segmentation IoU value are used as indicators to iteratively select the model parameters with the best generalization ability.

[0092] S5: Real-time Particle Segmentation Inference:

[0093] The grape thinning images to be segmented are input into the trained optimal model. After feature extraction and bi-branch enhancement, the output includes the grape outline, confidence level, and defect markers.

[0094] Compared with the prior art, the beneficial effects of the present invention are:

[0095] This invention utilizes a multi-layer bidirectional wavelet transform module to deconstruct and reconstruct features in the low-frequency to high-frequency domain, enhancing the gradient response of local details such as fruit edges and inter-fruit gaps, and improving edge visibility under complex lighting and occlusion scenarios. Through multi-path scanning and state-space recursion mechanisms of the visual state-space model, it captures the structural dependencies and long-distance spatial associations of the ellipsoidal morphology of the fruit bunch, enhancing the representation of the global structure of dense fruit. A dual-branch feature fusion mechanism achieves complementarity between spatial and frequency domain information, compensating for the shortcomings of traditional convolutional networks in capturing fine-grained features and overcoming the limitations of wavelet transform in lacking contextual modeling capabilities. Ultimately, while ensuring a lightweight model, it significantly improves the accuracy, robustness, and real-time inference efficiency of fruit segmentation during thinning, providing key technical support for grape phenotypic analysis and intelligent thinning decisions. Attached Figure Description

[0096] Figure 1 This is a schematic diagram of the wavelet transform combined with the visual state space model network structure of the present invention;

[0097] Figure 2 This is a schematic diagram illustrating the segmentation effect of the grape berry segmentation model and the comparative experimental model of the present invention. Detailed Implementation

[0098] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0099] A method for grape berry segmentation during thinning based on wavelet transform combined with a visual state space model, the method comprising the following steps:

[0100] S1: Constructing a grape berry segmentation dataset:

[0101] Based on the BBCH (Biologische Bundesanstalt, Bundessortenamt und Chemische Industrie, a unified coding system for plant growth cycles) of grapes, image data of key nodes from fruit setting to fruit thinning were collected. Targeted annotation of the target regions (grape berry region, berry surface defects, and bunch background) in the images was performed at the pixel level using professional annotation tools, forming a complete structured dataset containing image data and annotation information. This provides a high-quality data foundation for subsequent algorithm training and performance evaluation.

[0102] S2: Data Preprocessing and Augmentation

[0103] After performing standardized preprocessing on the dataset, it is divided into training set, validation set and test set according to a preset ratio. During the model training stage, online data augmentation strategy is used to dynamically generate diversified samples, expand the distribution range of training data, and improve the robustness and generalization ability of the model under different working conditions and various defect forms.

[0104] The online data augmentation strategies include one or more combinations of random rotation, scaling, mirroring, and cropping.

[0105] S3: The dual-branch augmentation network constructs and outputs global features F:

[0106] A backbone feature extraction network was constructed, consisting of a Stem module for initial feature extraction and a cascaded three-stage WT-Mamba phase module for enhancement.

[0107] For the semantic segmentation task of grape berry images during the thinning stage, a three-level WT-Mamba stage module is used to replace the single-path convolution or self-attention structure in the traditional coding network.

[0108] The three-level WT-Mamba stage module can gradually expand the receptive field, strengthen the contextual relationship, and capture the target morphological differences in grape growth stages at different semantic levels, thereby providing highly discriminative feature support for subsequent segmentation heads.

[0109] S4: Model Training and Optimization

[0110] The global feature F is input into the segmentation detection head for training. The model parameters are optimized by minimizing the difference between the predicted results and the ground truth annotations, allowing the model to gradually learn the discrimination features of grape targets under different scales and occlusions. During training, the validation set accuracy and segmentation IoU are used as indicators to iteratively select the model parameters with the best generalization ability.

[0111] S5: Real-time Particle Segmentation Inference:

[0112] The grape thinning images to be segmented are input into the trained optimal model. After feature extraction and bi-branch enhancement, the output includes the grape outline, confidence level, and defect markers.

[0113] Furthermore, the Stem module described in S3 adopts a simple architecture design, including three convolutional layers with 7×7 kernels connected in series and one convolutional layer with 3×3 kernels.

[0114] The 7×7 convolutional kernel, with its relatively large receptive field, can effectively capture low-level spatial features and is used to extract rich contextual information from the input image.

[0115] The 3×3 convolution kernel is used to perform feature refinement and spatial downsampling operations simultaneously.

[0116] This process reduces the resolution of the feature map, preserving key information while reducing computational complexity in subsequent stages.

[0117] In complex orchard environments, the large receptive field employed by the Stem module can effectively capture contextual information surrounding the grape berry structure, thereby efficiently extracting shallow texture features.

[0118] Furthermore, each level of the three-level WT-Mamba stage module described in S3 adopts a dual-branch parallel structure and fusion mechanism of MBWT (multi-scale bidirectional wavelet enhancement) branch and MambaEncoder (visual state space modeling) branch, which enhances the texture of fruit grain edges and long-distance spatial correlation while keeping the computational load controllable.

[0119] The MambaEncoder branch excels at modeling global structural relationships, while the MBWT branch excels at extracting local textures and edge details. During the fusion stage, the two are stacked element-wise, enabling the model to simultaneously utilize feature representations of different frequencies, scales, and structural dimensions. This achieves complementary enhancement of local fine textures and global topological information, significantly improving the detection and segmentation of densely packed small targets such as grape berries.

[0120] By merging the MambaEncoder branch and the MBWT branch in parallel, we have achieved efficient modeling of subtle edges, textures and long-range dependencies between the grape berry region and the vine background, which has significant structural innovation and application advantages.

[0121] Furthermore, step S3 includes the following steps:

[0122] S301: The Stem module receives a standardized grape image as input and performs preliminary extraction of low-level texture and edge features, as well as spatial dimension compression, through a combination of multiple convolutional layers; the input grape image... for:

[0123]

[0124] In the formula:

[0125] This indicates the batch size, which is the number of images input to the Stem module at one time.

[0126] This indicates the number of color channels in the input image;

[0127] These represent the height and width of the input image, respectively.

[0128] S302: The features output after processing by the Stem module flow sequentially through the three-level WT-Mamba stage module;

[0129] S303: Each level of the WT-Mamba stage generates multi-dimensional feature maps through the synergistic effect of MBWT enhancement and MambaEncoder, realizing deep semantic representation of grape berries, inflorescence structure and background environment. Finally, the feature maps output by the three-level WT-Mamba stage modules are fused to obtain the global feature F of the final output of the main body.

[0130] Furthermore, step S303 includes the following steps:

[0131] S30301: Perform Patch Embedding processing on the input features:

[0132] The two-dimensional features are divided into K×K non-overlapping image blocks by using a convolutional layer with a kernel size of K×K and a stride of K. Each image block is linearly projected and mapped to a fixed-dimensional feature token, which is then flattened into a one-dimensional sequence for processing by the state space model.

[0133] S30302: MBWT Branch Enhancement:

[0134] The output token sequence is input into the MBWT branch. The features are decomposed into LL low-frequency sub-band, LH horizontal high-frequency sub-band, HL vertical high-frequency sub-band and HH diagonal high-frequency sub-band through multi-layer bidirectional wavelet transform. After performing 1×1 convolution enhancement on each sub-band, cross-scale feature fusion is achieved by reconstructing through inverse Haar wavelet transform, highlighting the texture of the fruit edge and the gap between the fruit, which is helpful to distinguish the tightly clustered small fruit in complex background.

[0135] S30303: MambaEncoder Branch Modeling:

[0136] The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the Cross-Selective Scan module expands the features into eight sets of one-dimensional sequences along the rows, columns, diagonals and in the reverse direction. These sequences are then input into the continuous state space model for long-distance dependency modeling, capturing the structural relationship of the entire bunch of grapes and branches, and avoiding the fragmentation caused by focusing only on local areas.

[0137] S30304: Dual-branch fusion:

[0138] Features output by the MBWT branch Features of the output of the MambaEncoder branch Aligned to the same spatial dimension, the dual-branch feature fusion is achieved through pixel-by-pixel weighted summation, resulting in the final output features of the three-level WT-Mamba stage module.

[0139] This invention introduces the Visual Mamba State Space Model (VSSM) into feature encoding, constructing a dynamic matrix recursive network capable of performing multi-directional scan modeling in two-dimensional space. By generating eight scan sequences through Cross-SelectiveScan, the model can simultaneously capture spatial dependencies in the horizontal, vertical, and diagonal directions. Furthermore, it utilizes a learnable state transition matrix A, input / output mapping matrices B and C, and a fixed dimension to establish a dynamic modeling capability for long-distance structural dependencies.

[0140] This continuous modeling method based on state-space equations has the advantages of low computational cost and high scalability, and is particularly suitable for capturing key information such as the overall shape, arrangement direction, and occlusion relationship of grape berries in high-resolution grape images.

[0141] Furthermore, step S30301 includes the following steps:

[0142] S3030101: The input image is processed by a convolutional layer with a kernel size of K×K and a stride of K. Perform a patch convolution operation to obtain the output feature map. :

[0143]

[0144] In the formula:

[0145] This represents a convolution operation with a kernel size of K×K and a stride of K;

[0146] This indicates the number of channels output by Patch Embedding;

[0147] This indicates the height of the output image after Patch processing;

[0148] This indicates the width of the output image after Patch processing;

[0149] Indicates the padding parameters of the convolutional layer;

[0150] S3030102: Feature map implement Operations, its spatial dimensions Flatten into a token sequence :

[0151]

[0152] S3030103: Treat each spatial location as a token sequence, and for the token sequence... Perform a layer normalization operation to obtain the final output token sequence. :

[0153]

[0154] In the formula:

[0155] This indicates a normalization operation;

[0156] Indicates the number of tokens.

[0157] Furthermore, step S30302 includes the following steps:

[0158] S3030201: Multi-scale bidirectional Haar wavelet decomposition:

[0159] The input feature map is fed into the MBWT branch, and the input features are decomposed through multi-level bidirectional Haar wavelet transform (WT) to obtain multi-frequency scale features. ,in: This represents wavelet transform, used to achieve frequency and decomposition of features; The spatial size is half that of the original feature map, and the number of channels is... ; This represents the global channel percentage coefficient. This represents the original number of feature channels, with the coefficient 4 derived from one low-frequency component and three high-frequency components obtained from the Haar wavelet transform decomposition.

[0160] The Haar wavelet transform filters include low-pass filters. and high-pass filter bank , , ;

[0161] S3030202: Subband Local Convolution Enhancement:

[0162] Multi-frequency scale features Local convolution operations (Conv) are performed on each subband (low-frequency component LL, horizontal high-frequency component LH, vertical high-frequency component HL, and diagonal high-frequency component HH) to extract and optimize the feature information of each subband.

[0163] S3030203: Inverse Haar wavelet reconstruction and fusion:

[0164] Perform inverse Haar wavelet transform (IWT) on each sub-band feature after convolution enhancement to restore the processed frequency features to the original feature map size, thus obtaining the cross-scale fused features. ,in: This represents the inverse Haar wavelet transform, used to restore the processed frequency features to their original spatial dimensions.

[0165] low-pass filter Low-frequency structural information (corresponding to smooth areas of the image, reflecting the overall trend) is used to preserve features, while high-frequency noise is filtered out.

[0166] The high-pass filter bank is used to preserve high-frequency detail information of features (corresponding to the edges and textures of the image) to enhance the extraction of key local details. The wavelet branch explicitly separates low-frequency structure and high-frequency details at multiple scales to highlight the gradient response at the boundaries between fruit and background and between fruit and stem, improve the visibility of fruit edges under complex lighting and leaf occlusion, and help the subsequent decoder obtain more accurate boundary information.

[0167] Furthermore, step S30303 includes the following steps:

[0168] S3030301: Linear Projection and Two-Branch Partitioning:

[0169] For the input token sequence Perform a linear projection operation to map to a higher dimension G, resulting in two branch features. Its dimensions are Branch features As input to the gated branch, the branch features As input to the SSM (state space) branch;

[0170] S3030302: Gated branch weight generation:

[0171] branch features Applying the GELU activation function yields the gate weights. The subsequent SSM output is dynamically weighted to enhance the response to key fruit grain regions;

[0172] S3030303: Local feature enhancement of the SSM branch:

[0173] Branch features Rearranged into two-dimensional features according to spatial dimensions For two-dimensional features Perform depthwise convolution to capture local neighborhood relationships of the fruit particles, aggregate local texture and shape information, and obtain processed features. ,in: This represents a depthwise convolution function; it effectively enhances the local consistency of the grain region and extracts local patterns such as texture, spots, and bumps.

[0174] S3030304: Feature sequence expansion:

[0175] Two-dimensional features are processed using the CrossScan module. Expand along the rows, columns, diagonals, and their reverse directions to form eight one-dimensional sequences. ,in: ; Representing the The one-dimensional sequence obtained by group expansion, ;

[0176] S3030305: State-space recursive modeling:

[0177] For each group of one-dimensional sequences Perform continuous state-space recursive calculations, where:

[0178] The hidden state update formula is:

[0179]

[0180] The output formula is:

[0181]

[0182] In the formula:

[0183] Indicates the first The implicit state of a road;

[0184] Indicates the first The input sequence of the path;

[0185] These represent the learnable state transition matrix, input matrix, output matrix, and bypass mapping, respectively.

[0186] This structure enables the model to capture long-range dependencies in any direction and maintain global consistency even when the fruit particles are dense and occluded.

[0187] S3030306: Multi-path feature fusion:

[0188] Sequence of outputs from an eight-way state space Summation and fusion are performed to obtain a two-dimensional feature map after global state space modeling. ;

[0189] S3030307: Gated Fusion:

[0190] Gating weights With SSM branch output Perform an element-wise multiplication gated fusion operation to obtain the final output of the MambaEncoder branch. ,in: This is for element-wise multiplication (one-to-one correspondence between the same position and the same channel).

[0191] In the MambaEncoder branch, DWConv is used to perform detailed texture enhancement on each channel within the local grain region, strengthening local structures such as grain edges and gaps between grains. CrossScan stretches the entire image into a sequence along the horizontal, vertical, and diagonal lines, allowing the state-space model to capture long-range dependencies in these directions. Multi-path SSM output overlay allows the model to simultaneously perceive horizontal, vertical, and diagonal structural relationships, improving its ability to model overlapping grains. In the nonlinear gating branch, the GELU gating mechanism is used to amplify the long-range dependency information of the SSM by using a large gating value at grain edges and overlapping areas, while lowering the gating value in areas with interference such as pure background and light spots to prevent the SSM from propagating noise further.

[0192] The features obtained by using the MambaEncoder branch have both multi-directional long-range dependencies (which are beneficial for identifying the connectivity structure of the whole bunch of grapes) and local departmental noise suppression (which reduces interference from background elements such as leaves and branches), which significantly helps to improve the integrity of the grape berry segmentation outline and the separation of clumps of grapes during the thinning period.

[0193] The MBWT branch introduced in the WT-Mamba stage can perform multi-scale, bidirectional decomposition of input features using orthogonal wavelet bases before the feature map enters deep modeling. This decomposition process captures local high-frequency features of grape berry edges, highlights, bloom textures, and gaps between berries at different spatial frequencies, and achieves learnable frequency-selective enhancement through sub-band convolution, thereby strengthening the structural information of berry boundaries and occluded areas and improving the ability of subsequent segmentation models to distinguish dense berries.

[0194] Furthermore, the three-level WT-Mamba stage module described in S303 outputs feature maps with different spatial resolutions and semantic levels, specifically:

[0195] Feature map C2: Output from the first WT-Mamba stage, with the highest spatial resolution and rich details of grain texture;

[0196] Feature map C3: Output from the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution;

[0197] Feature map C5: Output from the third WT-Mamba stage, containing the highest-level granule semantic structure information.

[0198] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0199] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for grape berry segmentation in thinning stage based on wavelet transform combined with visual state space model, characterized in that: The method comprises the following steps: S1: Constructing a grape berry segmentation dataset: According to the BBCH of the grape growth cycle, image data of the fruit thinning period from the fruit setting period to the fruit development period is targetedly collected, pixel-level labeling of the target region in the image is performed through a professional labeling tool, and a complete structured dataset containing image data and labeling information is formed; S2: Data preprocessing and enhancement: After standardizing preprocessing of the dataset, the dataset is divided into a training set, a validation set and a test set according to a preset ratio, online data enhancement strategies are adopted in the model training stage to dynamically generate diversified samples, and the distribution range of the training data is expanded; S3: Building a double-branch enhanced network and outputting global features F: A backbone feature extraction network including a Stem module for initial feature extraction and a series of three-stage WT-Mamba stage modules for enhancement is built; S4: Model training and optimization: The global features F are input into a segmentation detection head for training, the model parameters are optimized by minimizing the difference between the predicted results and the real labels, and the generalization ability of the model parameters is iteratively screened in the training process according to the validation set accuracy and the segmentation IoU value as indexes; S5: Real-time berry segmentation reasoning: The grape image to be segmented in the fruit thinning period is input into the trained optimal model, and the segmentation results containing the berry contour, confidence and defect label are output after feature extraction and double-branch enhancement.

2. The grape berry segmentation method in the fruit thinning period based on wavelet transform combined with a visual state space model according to claim 1, characterized in that: The Stem module in S3 comprises three 7x7 convolution kernel convolution layers connected in series and a 3x3 convolution kernel convolution layer; The 7x7 convolution kernel captures low-level spatial features and is used to extract context information from the input image; The 3x3 convolution kernel is used to perform feature refinement and spatial down-sampling operations simultaneously.

3. The method of claim 2, wherein the method is a method of grape berry segmentation during thinning period based on wavelet transform combined with visual state space model. Each WT-Mamba stage of the three-stage WT-Mamba stage module in S3 adopts a double-branch parallel structure of an MBWT branch and a MambaEncoder branch.

4. The method of claim 3, wherein the method is a method of grape berry segmentation during thinning period based on wavelet transform combined with visual state space model. S3 comprises the following steps: S301: The stem module receives the input grape image, and completes preliminary extraction of bottom texture and edge features and spatial dimension compression through combination of multiple convolution layers. is: In the formula: represents the number of images inputted at one time into the Stem module; represents the number of color channels of the input image; respectively represent the height and width of the input image; S302: The features output after the Stem module processing flow through the three-stage WT-Mamba stage module in turn; S303: Each WT-Mamba stage generates a multi-dimensional feature map through the synergistic effect of MBWT enhancement and MambaEncoder, and finally fuses the feature maps output by the three-stage WT-Mamba stage module to obtain the global features F output by the backbone; 5. The method for grape berry thinning period segmentation based on wavelet transform combined with visual state space model according to claim 4, characterized in that: S303 comprises the following steps: S30301: Perform Patch Embedding processing on the input features: Divide the two-dimensional features into KxK non-overlapping image blocks through a convolution layer with a convolution kernel size of KxK and a step size of K, perform linear projection on each image block, map it to a fixed-dimensional feature token, and flatten it into a one-dimensional sequence form for state space model processing; S30302: MBWT branch enhancement: The output token sequence is input into the MBWT branch, and the features are decomposed into LL low-frequency subbands, LH horizontal high-frequency subbands, HL vertical high-frequency subbands and HH diagonal high-frequency subbands through multi-layer bidirectional wavelet transform. After 1x1 convolution enhancement is performed on each subband, cross-scale feature fusion is realized through inverse Haar wavelet reconstruction, highlighting the grain edge and inter-grain gap texture; S30303: MambaEncoder branch modeling: The output token sequence is input into the MambaEncoder branch, and the effective features are filtered through the gating mechanism. Then, the features are unfolded along the row, column, diagonal and reverse direction into eight one-dimensional sequences through the Cross-Selective Scan module, and input into the continuous state space model for long-distance dependence modeling. S30304: Dual-branch fusion: Features output by the MBWT branch Features output by the MambaEncoder branch Aligning to the same spatial dimension, the dual-branch feature fusion is realized by pixel-wise weighted sum, and the final output feature of the three-level WT-Mamba stage module is obtained.

6. The method for grape berry thinning period segmentation based on wavelet transform combined with visual state space model according to claim 5, characterized in that: The S30301 includes the following steps: S3030101: performing convolution operation on the input image by a convolution layer with a kernel size of KxK and a step size of K performing patch convolution operation to obtain an output feature map : In the formula: denotes a convolution operation with kernel size K x K and stride K; number of channels representing the Patch Embedding output; represents the height of the output image after patch processing; represents the width of the output image after Patch processing; padding parameter representing a convolution layer; S3030102: to feature map performing operations, flattening its spatial dimensions into a sequence of tokens : S3030103: on the token sequence performing layer normalization operation to obtain the final output token sequence : In the formula: denotes a normalization operation; represents the number of tokens.

7. The method for grape berry thinning period segmentation based on wavelet transform combined with visual state space model according to claim 6, characterized in that: The S30302 includes the following steps: S3030201: Multi-scale bidirectional Haar wavelet decomposition: The input feature map is input into the MBWT branch, and the input feature is decomposed through multi-layer bidirectional Haar wavelet transform to obtain multi-frequency scale features wherein: indicates a wavelet transform, and is used to realize frequency and decomposition of the feature; The spatial size of the feature is 1 / 2 of the original feature map, and the number of channels is ; indicates a global channel proportion coefficient, indicates the number of original feature channels, and the coefficient 4 is derived from 1 low-frequency component and 3 high-frequency components obtained through the Haar wavelet transform decomposition; The filter of the Haar wavelet transform comprises a low-pass filter and a high-pass filter set , , ; S3030202: Subband local convolution enhancement: Multi-frequency scale features The local convolution operation is performed on each sub-band respectively, and the feature information of each sub-band is extracted and optimized. S3030203: Inverse Haar wavelet reconstruction fusion: An inverse Haar wavelet transform is performed on the convolution-enhanced sub-band features to restore the processed frequency features to the original feature map size, to obtain cross-scale fused features wherein: denotes an inverse Haar wavelet transform for restoring the processed frequency features to the original spatial size; Low pass filter For preserving low frequency structural information of features, filtering high frequency noise; The high-pass filter bank is used to retain the high-frequency detail information of the features, realize the enhancement extraction of the key local details, highlight the gradient response at the fruit grain-background and fruit grain-fruit stem junction, and improve the visibility of the fruit grain edge under complex illumination and leaf shielding.

8. The method for grape berry thinning period segmentation based on wavelet transform combined with visual state space model according to claim 7, characterized in that: The S30303 includes the following steps: S3030301: Linear projection and dual-branch division: For an input token sequence performing a linear projection operation to map to a higher dimensional G, resulting in two branch features with dimensions concatenating the branch features as input to the gating branch, and the branch features as input to the SSM branch; S3030302: Gating branch weight generation: Branching features Applying a GELU activation function to the gating weights ; S3030303: Local feature enhancement of SSM branch: Branching features are identified Re-arranging the spatial dimensions into two-dimensional features , two-dimensional features are identified Performing deep convolution operations to capture local neighborhood relations of the puffs, aggregate local texture and shape information, and obtain processed features wherein: represents a deep convolution function; S3030304: Feature sequence unfolding: Two-dimensional features are spread into one-dimensional sequences by the CrossScan module Along rows, columns, diagonals and their inverses into eight groups of one-dimensional sequences Where: ; Represents the one-dimensional sequence obtained from the first group spread, ; S3030305: State space recursive modeling: for each set of one-dimensional sequences performing a continuous state space recursion computation, wherein: The hidden state update formula is: The output formula is: In the formula: represents the implicit state of the path; represents the input sequence of the path; respectively denote a learnable state transition matrix, an input matrix, an output matrix, and a bypass mapping; S3030306: Multi-path feature fusion: sequences of eight-way state space outputs sum fusion to obtain a two-dimensional feature map after global state space modeling ; S3030307: Gating fusion: Gating weights with the SSM branch output performing element-wise multiplication to get the final output of the MambaEncoder branch wherein: is element-wise multiplication.

9. The method of claim 4 or 8, wherein the method is a method of grape berry thinning period berry segmentation based on wavelet transform combined with visual state space model. The three-level WT-Mamba stage modules output feature maps with different spatial resolutions and semantic levels, specifically: Feature map C2: Output by the first WT-Mamba stage, with the highest spatial resolution, containing fruit grain texture details; Feature map C3: Output by the second WT-Mamba stage, achieving a balance between semantic abstraction and spatial resolution; Feature map C5: Output by the third WT-Mamba stage, containing the highest level of fruit grain semantic structure information.

Images