Method for segmenting grape leaf disease area image in coordination of frequency domain and spatial domain
By using the global processing module and the hierarchical feature constructor in tandem, the problems of background interference and scale variation in grape leaf disease area segmentation were solved, and high-accuracy segmentation of disease areas was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHONGQING NORMAL UNIVERSITY
- Filing Date
- 2025-11-14
- Publication Date
- 2026-05-29
AI Technical Summary
Existing technologies suffer from insufficient segmentation accuracy in grape leaf disease area segmentation due to interference from complex field environment backgrounds and significant scale differences in disease characteristics. In particular, convolutional neural networks and Transformer architectures are not effective in handling multi-scale distributions and complex spatial structure features.
A global processing module is adopted, which synchronously performs frequency domain transformation and spatial selective aggregation within the window through a frequency-spatial domain collaborative attention mechanism. Combined with a hierarchical feature builder and an adaptive feature fusion module, it enhances feature discrimination ability and suppresses background interference, and handles significant scale changes in disease areas.
It improves the accuracy of segmentation of diseased areas on grape leaves, effectively integrates multi-scale heterogeneous features, enhances the feature fusion capability of the network, and ensures accurate segmentation of diseased areas.
Smart Images
Figure CN122116089A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an image segmentation method, and more particularly to an image segmentation method for diseased areas of grape leaves using both frequency and spatial domains. Background Technology
[0002] Grape leaves are susceptible to various pathogens during growth, leading to diseases that affect the plant's photosynthesis and nutrient metabolism, resulting in decreased fruit quality and ultimately severe yield losses and economic damage. Traditional methods for identifying grape leaf diseases rely mainly on expert observation and experience, which are limited by low efficiency, high subjectivity, and difficulty in large-scale application.
[0003] With the development of computer vision technology, disease identification based on grape leaf images has been widely applied. To ensure recognition accuracy, it is necessary to accurately segment the diseased regions in grape leaf images. However, in existing technologies, background interference in complex field environments and significant scale differences in disease features limit the accuracy of grape leaf disease region segmentation. For example, image segmentation methods based on convolutional neural networks have the following drawbacks: localized connection mechanisms are difficult to establish long-distance dependencies, making it difficult to fully capture the global semantic information of complex disease patterns; fixed convolutional kernels and weight parameters easily lead to the loss of detailed information, failing to effectively represent the pixel-level semantic features of small lesion areas; and this is especially problematic for grape leaf diseases with multi-scale distribution and complex spatial structure features. Another example is the global attention scheme based on the Transformer architecture. Although the cross-resolution Transformer architecture and context mining mechanism effectively integrate feature information at different resolution levels, the influence of irrelevant information in complex backgrounds leads to misjudgment of key information of disease features, resulting in reduced accuracy of the final image segmentation result.
[0004] Therefore, in order to solve the above-mentioned technical problems, it is urgent to propose a new technical approach. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide a method for image segmentation of grape leaf disease areas in a coordinated frequency and spatial domain. The image segmentation model employs a global processing module, which, through a frequency-spatial domain coordinated attention mechanism, simultaneously performs frequency domain transformation and spatial selective aggregation within a window. This achieves deep coordination between global frequency domain perception and precise spatial localization, enhancing feature discrimination capabilities while effectively suppressing interference from complex field backgrounds on disease feature extraction. Furthermore, a hierarchical feature builder is used during feature extraction to capture fast-scale contextual information hierarchically, effectively handling significant scale variations in disease areas. Additionally, the adaptive feature fusion module can adaptively integrate multi-scale heterogeneous features dynamically, enhancing the network's feature fusion capability, thereby effectively ensuring the final accuracy of grape leaf disease area segmentation.
[0006] This invention provides a method for image segmentation of diseased areas on grape leaves using both frequency and spatial domains, comprising the following steps:
[0007] S1. Acquire grape leaf sample image information and preprocess the grape leaf sample image information;
[0008] S2. Construct an image segmentation model based on the co-domain and spatial domain. The image segmentation model includes an input convolution module, a hierarchical feature builder for multi-level feature extraction, an adaptive feature fusion module, a global processing module, and a segmentation head.
[0009] The input convolution module is used to perform preliminary feature extraction processing on the input image. The output features of the input convolution module are input into the hierarchical feature builder. The hierarchical feature builder outputs features at four different scales, namely F1, F2, F3 and F4.
[0010] Features F1, F2, and F3 are all input into the adaptive feature fusion module. The output features of the adaptive feature fusion module are input into the global processing module. The global processing module outputs four features, namely FO1, FO2, FO3, and FO4. Feature FO1 is added element-wise with feature F2 and then input into the segmentation head. Feature FO2 is added element-wise with feature F3 and then input into the segmentation head. Feature FO3 is added element-wise with feature F4 and then input into the segmentation head. Feature F4 is also input into the segmentation head. Feature F4 is also input into the average pooling module. The output features of the average pooling module are input into the segmentation head. The segmentation head outputs the segmentation result.
[0011] The preprocessed grape leaf sample image information is input into the image segmentation model for training;
[0012] S3. Obtain the real-time image information of the grape leaves to be segmented, and input the preprocessed real-time image information of the grape leaves into the trained image segmentation model to obtain the segmentation result.
[0013] Furthermore, the hierarchical feature constructor includes a first-stage processing module, a second-extreme processing module, a third-stage processing module, and a fourth-stage processing module;
[0014] The first-stage processing module includes two cascaded gated convolutional units, and the first-stage processing module outputs feature F1; the second-stage processing module includes three cascaded gated convolutional units, and the second-stage processing module outputs feature F2; the third-stage processing module includes three cascaded multi-scale perception modules, and the third-stage processing module outputs feature F3; the fourth-stage processing module includes two multi-scale perception modules, and the fourth-stage processing module outputs feature F4; the gated convolutional unit structures of the first-stage and second-stage processing modules are the same; the multi-scale perception module structures of the third-stage and fourth-stage processing modules are the same.
[0015] Furthermore, the gated convolutional unit includes a convolution module I, a segmentation module I, a depthwise convolution module I, a batch normalization module I, a ReLU6 activation function module I, a splicing module I, a convolution module II, a batch normalization module II, and a GELU activation function module I;
[0016] The input of convolution module I serves as the input of the gated convolutional unit. The output features of convolution module I are input to segmentation module I, which divides the input features into three features along the channel direction: feature GF1, feature GF2, and feature GF3. Feature GF2 is input to depth convolution module I, and the output features of depth convolution module I are input to batch normalization module I. The output features of batch normalization module I are input to ReLU6 activation function module I, and the output features of ReLU6 activation function module I are input to concatenation module I. Concatenation module I concatenates feature GF1 and the output features of ReLU6 activation function module I. Feature GF3 is input to GELU activation function module I. The output features of GELU activation function module I are multiplied element-wise with the output features of concatenation module I and then input to convolution module II. The output features of convolution module II are input to batch normalization module II, and the output features of batch normalization module II are added element-wise with the input features of convolution module I to serve as the output features of the gated convolutional unit.
[0017] Furthermore, the multi-scale perception module includes a convolution module III, a segmentation module II, a depthwise convolution module II, a depthwise convolution module III, a depthwise convolution module IV, a batch normalization module III, a batch normalization module IV, a batch normalization module V, a batch normalization module VI, a batch normalization module VII, a ReLU6 activation function module II, a ReLU6 activation function module III, a ReLU6 activation function module IV, a ReLU6 activation function module V, a stitching module II, a convolution module IV, a convolution module V, a convolution module VI, and a sigmoid function module;
[0018] The input of convolutional module III serves as the input of the multi-scale perception module. The output features of convolutional module III are input to segmentation module II, which divides the input features into three equal parts along the channel direction: feature MF1, feature MF2, and feature MF3. Feature MF1 is input to deep convolutional module II, and the output features of deep convolutional module II are input to batch normalization module III. The output features of batch normalization module III are then input to ReLU6 activation function module II. Feature MF2 is input to deep convolutional module III, and the output features of deep convolutional module III are input to batch normalization module IV. The output features of batch normalization module IV are then input to ReLU6 activation function module III. Feature MF3 is input to deep convolutional module IV, and the output features of deep convolutional module IV are input to batch normalization module V. The output features of batch normalization module V are then input to ReLU6 activation function module IV. The dilation coefficients of deep convolutional modules II, III, and IV are 1, 3, and 5, respectively.
[0019] The output features of RELU6 activation function modules II, III, and IV are all input into concatenation module II. The output features of concatenation module II are input into convolution module IV. The input and output features of convolution module IV are input into batch normalization module VI. The output features of batch normalization module VI are input into RELU6 activation function module V. The output features of RELU6 activation function module V are input into convolution module V. The output features of convolution module V are input into batch normalization module VII. The output features of batch normalization module VII are input into sigmoid function module. The output features of sigmoid function module are multiplied element-wise with the output features of concatenation module II and then input into convolution module VI. The output features of convolution module VI are added element-wise with the input features of convolution module III to obtain the final output result of the multi-scale perception module.
[0020] Furthermore, the global processing module includes a global processing module I and a global processing module II with identical structures and connected in series; the global processing module I adopts a no-offset window mode, and the global processing module II adopts a sliding window mode.
[0021] The global processing module I includes a window partitioning module, a frequency domain spatial collaborative attention module, a window restoration module, a convolution module VIII, a convolution module IX, a depthwise convolution module V, a convolution module attention module, and a feedforward neural network layer;
[0022] The input features of global processing module I are input into the window partitioning module and convolution module VIII, respectively. The output features of the window partitioning module are input into the frequency domain spatial collaborative attention module. The output features of the frequency domain spatial collaborative attention module are input into the window restoration module. The output features of convolution module VIII are input into the deep convolution module V. The output features of deep convolution module V are input into the convolution module attention module. The output features of the convolution module attention module are input into convolution module IX. The output features of convolution module IX and window restoration module are summed element-wise and then input into the feedforward neural network layer. The output of the feedforward neural network layer serves as the output of global processing module I.
[0023] Furthermore, the frequency-domain spatial-domain collaborative attention module includes linear layer I, linear layer II, linear layer III, Fourier transform module, basic frequency-domain filter, global average pooling layer, Act function module, inverse Fourier transform module, and softmax module I;
[0024] The input window partitioning module of the linear layer I, linear layer II and linear layer III outputs the features, with linear layer I outputting the value feature V, linear layer II outputting the query feature Q, and linear layer III outputting the key feature K;
[0025] The value feature V is input to the Fourier transform module and the global average pooling layer. The global average pooling layer outputs modulation parameters. The value feature V is also input to the base filter. The modulation parameters are input to the base filter. The output feature of the base filter is multiplied element-wise with the output feature of the Fourier transform module and then input to the Act function module. The output feature of the Act function module is input to the inverse Fourier transform module. The query feature Q and the key feature K are multiplied and then input to the softmax module I. The output feature of the softmax module I is multiplied with the output feature of the inverse Fourier transform module and then used as the output feature of the frequency-domain spatial-domain collaborative attention module.
[0026] Furthermore, the output end of the global processing module is also equipped with upsampling module I, upsampling module II, upsampling module III and upsampling module IV; upsampling module I, upsampling module II, upsampling module III and upsampling module IV all input the same feature, upsampling module I outputs feature F01, upsampling module II outputs feature F02, upsampling module III outputs feature F03 and upsampling module IV outputs feature F04.
[0027] Furthermore, the segmentation head includes a stitching module III, a convolution module X, a convolution module XI, a batch normalization module VIII, and a ReLU6 activation function module VI;
[0028] The input of the stitching module III serves as the input of the segmentation head. The output of the stitching module III is connected to the input of the convolution module X. The output of the convolution module X is connected to the input of the ReLU6 activation function module VI. The output of the ReLU6 activation function module VI is connected to the input of the convolution module XI. The output of the convolution module XI serves as the output of the segmentation head, outputting the image segmentation result.
[0029] Furthermore, the adaptive feature fusion module includes a downsampling module, a concatenation module IV, a convolution module XII, a ReLU activation function module, a convolution module XIII, a softmax module II, and a concatenation module V;
[0030] The downsampling module performs downsampling operations on features F1, F2, F3 and F4 respectively to form features F1', F2', F3' and F4';
[0031] Features F1', F2', F3', and F4' are input into the stitching module IV. The output features of stitching module IV are input into convolution module XII. The output features of convolution module XII are input into the ReLU activation function module. The output features of the ReLU activation function module are input into convolution module XIII. The output features of convolution module XIII are input into softmax module II. Softmax module II outputs feature weights, which correspond one-to-one with features F1', F2', F3', and F4'. Features F1', F2', F3', and F4' are multiplied by their corresponding feature weights and then input into stitching module V. The output features of stitching module V serve as the output of the adaptive feature fusion module.
[0032] The beneficial effects of this invention are as follows: By employing a global processing module in the image segmentation model, this invention utilizes a frequency-spatial-domain collaborative attention mechanism to simultaneously perform frequency domain transformation and spatial selective aggregation within a window. This achieves deep collaboration between global frequency domain perception and precise spatial positioning, enhancing feature discrimination capabilities while effectively suppressing interference from complex field backgrounds on disease feature extraction. Furthermore, a hierarchical feature builder is used during feature extraction to capture fast-scale contextual information hierarchically, effectively handling significant scale changes in diseased areas. Moreover, the adaptive feature fusion module can adaptively integrate multi-scale heterogeneous features dynamically, enhancing the network's feature fusion capability. This effectively ensures the final accuracy of grape leaf disease segmentation. Attached Figure Description
[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0034] Figure 1 This is a schematic diagram of the process of the present invention.
[0035] Figure 2 This is a schematic diagram of the image segmentation model structure of the present invention.
[0036] Figure 3 This is a schematic diagram of the layered feature constructor structure of the present invention.
[0037] Figure 4 This is a schematic diagram of the gated convolutional unit structure of the present invention.
[0038] Figure 5 This is a schematic diagram of the multi-scale sensing module structure of the present invention.
[0039] Figure 6 This is a schematic diagram of the segmentation head structure of the present invention.
[0040] Figure 7 This is a schematic diagram of the global processing module structure of the present invention.
[0041] Figure 8 This is a schematic diagram of the frequency domain and spatial domain collaborative attention module structure of the present invention.
[0042] Figure 9 This is a schematic diagram of the adaptive feature fusion module structure of the present invention. Detailed Implementation
[0043] The present invention will be further described in detail below:
[0044] This invention provides a method for image segmentation of diseased areas on grape leaves using both frequency and spatial domains, comprising the following steps:
[0045] S1. Acquire grape leaf sample image information and preprocess the grape leaf sample image information; wherein, the preprocessing includes denoising the sample image;
[0046] S2. Construct an image segmentation model based on the co-domain and spatial domain. The image segmentation model includes an input convolution module, a hierarchical feature builder for multi-level feature extraction, an adaptive feature fusion module, a global processing module, and a segmentation head.
[0047] The input convolution module is used to perform preliminary feature extraction processing on the input image. The output features of the input convolution module are input into the hierarchical feature builder. The hierarchical feature builder outputs features at four different scales, namely F1, F2, F3 and F4.
[0048] Features F1, F2, and F3 are all input into the adaptive feature fusion module. The output features of the adaptive feature fusion module are input into the global processing module. The global processing module outputs four features, namely FO1, FO2, FO3, and FO4. Feature FO1 is added element-wise with feature F2 and then input into the segmentation head. Feature FO2 is added element-wise with feature F3 and then input into the segmentation head. Feature FO3 is added element-wise with feature F4 and then input into the segmentation head. Feature F4 is also input into the segmentation head. Feature F4 is also input into the average pooling module. The output features of the average pooling module are input into the segmentation head. The segmentation head outputs the segmentation result.
[0049] The preprocessed grape leaf sample image information is input into the image segmentation model for training. When training the image segmentation model, the existing cross-entropy loss function is used for parameter update training. The cross-entropy loss function is an existing technology and will not be described in detail here.
[0050] S3. Obtain real-time image information of the grape leaves to be segmented. After preprocessing the real-time image information of the grape leaves, input it into the trained image segmentation model to obtain the segmentation result. Using the above method, a global processing module is adopted in the image segmentation model. This module, through a frequency-spatial-domain collaborative attention mechanism, simultaneously performs frequency domain transformation and spatial selective aggregation within a window, achieving deep collaboration between global frequency domain perception and precise spatial localization. This enhances feature discrimination capabilities while effectively suppressing the interference of complex field backgrounds on disease feature extraction. Furthermore, a hierarchical feature builder is used during feature extraction to capture fast-scale contextual information hierarchically, effectively handling significant scale changes in diseased areas. The adaptive feature fusion module can adaptively integrate multi-scale heterogeneous features dynamically, enhancing the network's feature fusion capability, thereby effectively ensuring the final accuracy of grape leaf disease segmentation.
[0051] The input convolution module consists of two concatenated convolutional layers.
[0052] In this embodiment, the hierarchical feature constructor includes a first-stage processing module, a second-stage processing module, a third-stage processing module, and a fourth-stage processing module connected in sequence. Here, the sequential connection means that the input and output are connected in sequence, that is, the output of the first-stage processing module is connected to the input of the second-stage processing module, the output of the second-stage processing module is connected to the input of the third-stage processing module, and the output of the third-stage processing module is connected to the input of the fourth-stage processing module.
[0053] The first-stage processing module includes two cascaded gated convolutional units, and its output feature F1; the second-stage processing module includes three cascaded gated convolutional units, and its output feature F2; the third-stage processing module includes three cascaded multi-scale perception modules, and its output feature F3; the fourth-stage processing module includes two multi-scale perception modules, and its output feature F4. The gated convolutional unit structures of the first and second-stage processing modules are identical; the multi-scale perception module structures of the third and fourth-stage processing modules are identical. Specifically, the first and second-stage processing modules perform shallow feature extraction using gated convolutional units, which efficiently extract shallow features and achieve adaptive feature selection through a gating mechanism. The third and fourth-stage processing modules are used for deep feature extraction, employing multi-scale perception modules to effectively capture cross-scale contextual information and effectively handle significant scale changes in the diseased area.
[0054] In this embodiment, the gated convolutional unit includes a convolution module I, a segmentation module I, a depthwise convolution module I, a batch normalization module I, a ReLU6 activation function module I, a splicing module I, a convolution module II, a batch normalization module II, and a GELU activation function module I;
[0055] The input of convolution module I serves as the input of the gated convolution unit. The output features of convolution module I are input to segmentation module I, which divides the input features into three features along the channel direction: feature GF1, feature GF2, and feature GF3. Feature GF2 is input to depthwise convolution module I, and the output features of depthwise convolution module I are input to batch normalization module I. The output features of batch normalization module I are input to ReLU6 activation function module I, and the output features of ReLU6 activation function module I are input to concatenation module I. Concatenation module I concatenates feature GF1 and the output features of ReLU6 activation function module I. Feature GF3 is input to GELU activation function module I, and GELU activation... The output features of live function module I are multiplied element-wise with the output features of concatenation module I and then input into convolution module II. The output features of convolution module II are input into batch normalization module II. The output features of batch normalization module II are added element-wise with the input features of convolution module I and then used as the output features of the gated convolutional unit. In the above, both convolution module I and convolution module II use 1×1 convolution kernels. Since the output features of concatenation module I differ in channel dimension from the output features of GF3 processed by the GELU function module, an upsampling module is generally set at the output of the GELU function module to ensure that the output features of concatenation module I have the same dimension when multiplied with the output features of the GELU function module. Figure 4 In this context, UPSamp represents the upsampling module.
[0056] In this embodiment, the multi-scale perception module includes a convolution module III, a segmentation module II, a depthwise convolution module II, a depthwise convolution module III, a depthwise convolution module IV, a batch normalization module III, a batch normalization module IV, a batch normalization module V, a batch normalization module VI, a batch normalization module VII, a ReLU6 activation function module II, a ReLU6 activation function module III, a ReLU6 activation function module IV, a ReLU6 activation function module V, a stitching module II, a convolution module IV, a convolution module V, a convolution module VI, and a sigmoid function module;
[0057] The input of convolutional module III serves as the input of the multi-scale perception module. The output features of convolutional module III are input to segmentation module II, which divides the input features into three equal parts along the channel direction: features MF1, MF2, and MF3. Feature MF1 is input to deep convolutional module II, and its output features are input to batch normalization module III, which in turn are input to ReLU6 activation function module II. Feature MF2 is input to deep convolutional module III, and its output features are input to batch normalization module IV, which in turn are input to ReLU6 activation function module III. Feature MF3 is input to deep convolutional module IV, and its output features are input to batch normalization module V, which in turn are input to ReLU6 activation function module IV. The dilation coefficients (also known as dilation rates or dilation scales) of deep convolutional modules II, III, and IV are 1, 3, and 5, respectively.
[0058] The output features of RELU6 activation function modules II, III, and IV are all input into concatenation module II. The output features of concatenation module II are input into convolution module IV. The input and output features of convolution module IV are input into batch normalization module VI. The output features of batch normalization module VI are input into RELU6 activation function module V. The output features of RELU6 activation function module V are input into convolution module V. The output features of convolution module V are input into batch normalization module VII. The output features of batch normalization module VII are input into sigmoid function module. The output features of sigmoid function module are multiplied element-wise with the output features of concatenation module II and then input into convolution module VI. The output features of convolution module VI are added element-wise with the input features of convolution module III to obtain the final output result of the multi-scale perception module. Among them, convolutional module III, convolutional module IV, convolutional module V, and convolutional module VI all use 1×1 convolutional kernels;
[0059] The input features of the multi-scale perception module are processed by convolution module III. Then, the segmentation module II converts the output features of convolution module III into three features along the channel direction and inputs them into three depth convolution modules. Each convolution module has a different dilation coefficient. After the three features are processed by depth-separable convolution with the corresponding dilation rate, they are concatenated along the channel dimension to form a multi-scale feature representation. Then, the concatenation module II performs feature fusion and then uses a multi-scale mechanism (i.e., after processing through the branches of convolution modules IV and V, it is fused with the output features of concatenation module II) to achieve the capture of complex contextual information.
[0060] In this embodiment, the global processing module includes a global processing module I and a global processing module II with identical structures and connected in series; the global processing module I adopts a no-offset window mode, and the global processing module II adopts a sliding window mode.
[0061] The global processing module I includes a window partitioning module, a frequency domain spatial collaborative attention module, a window restoration module, a convolution module VIII, a convolution module IX, a depthwise convolution module V, a convolution module attention module, and a feedforward neural network layer;
[0062] The input features of global processing module I are input into the window partitioning module and convolution module VIII, respectively. The output features of the window partitioning module are input into the frequency domain spatial collaborative attention module. The output features of the frequency domain spatial collaborative attention module are input into the window restoration module. The output features of convolution module VIII are input into the deep convolution module V. The output features of deep convolution module V are input into the convolution module attention module. The output features of the convolution module attention module are input into convolution module IX. The output features of convolution module IX and window restoration module are summed element-wise and then input into the feedforward neural network layer. The output of the feedforward neural network layer serves as the output of global processing module I.
[0063] The window partitioning and window restoration modules both employ existing technologies, such as those found in the existing Transformer system. In the global processing module I, the window partitioning module divides the feature map into windows without any overlap between elements in the feature sub-maps within each window, meaning there is no offset during window partitioning. In the global processing module II, a sliding window approach is used for window partitioning, with the window size and sliding stride obtained during training. The convolutional kernels of convolutional modules VIII and IX have a scale of 1×1. The feedforward neural network layer also uses existing technologies and will not be elaborated upon here.
[0064] Wherein: the frequency domain spatial domain collaborative attention module includes linear layer I, linear layer II, linear layer III, Fourier transform module, basic frequency domain filter, global average pooling layer, Act function module, inverse Fourier transform module, and softmax module I;
[0065] The input window partitioning module of the linear layer I, linear layer II and linear layer III outputs the features, with linear layer I outputting the value feature V, linear layer II outputting the query feature Q, and linear layer III outputting the key feature K;
[0066] The value feature V is input to the Fourier transform module and the global average pooling layer. The global average pooling layer outputs modulation parameters. The value feature V is also input to the base filter. The modulation parameters are input to the base filter. The output feature of the base filter is multiplied element-wise with the output feature of the Fourier transform module and then input to the Act function module. The output feature of the Act function module is input to the inverse Fourier transform module. The query feature Q and the key feature K are multiplied and then input to the softmax module I. The output feature of the softmax module I is multiplied with the output feature of the inverse Fourier transform module and then used as the output feature of the frequency-domain spatial-domain collaborative attention module.
[0067] In the frequency-domain and spatial-domain collaborative attention module, after the window partitioning module processes and partitions the window feature X, a query feature Q, a key feature K, and a value feature V are first generated through a linear layer. The value feature V enters the frequency domain path, while the query feature Q and the key feature K enter the spatial path. A global average pooling layer then extracts information from the value feature parameters and generates modulation parameters. The modulation parameters are... and Then the modulation parameters are adjusted against the basic filter parameters. Modulation is performed to achieve fine modulation of different frequency bands, forming the output characteristics FL of the basic filter, specifically: The output features FL of the basic filter are multiplied element-wise with the output features F(V) of the Fourier transform module; then the Act activation function is applied.
[0068] ;in: ;
[0069] The basic filter is an existing technology and can be the basic filter in the Transformer network. Then, the inverse Fourier transform module performs an inverse transform on the output features of the Act function module. The softmax module I outputs the corresponding weights, which are then multiplied element-wise with the output features of the inverse Fourier transform module to obtain the final output result of the frequency domain spatial domain collaborative attention module.
[0070] The branch context enhancement path, consisting of convolutional module VIII, convolutional module IX, depthwise convolutional module V, and convolutional module attention module, first projects the input features through a 1×1 convolution (i.e., convolutional module VIII), then applies a depthwise separable convolution (i.e., depthwise convolutional module V) for spatial feature extraction, and finally uses the convolutional module attention module (CBAM module, which is the existing technology) for feature extraction.
[0071] The feedforward neural network layer is existing technology and will not be described in detail here.
[0072] In this embodiment, the output end of the global processing module is further provided with upsampling module I, upsampling module II, upsampling module III, and upsampling module IV. Upsampling module I, upsampling module II, upsampling module III, and upsampling module IV all input the same feature. Upsampling module I outputs feature F01, upsampling module II outputs feature F02, upsampling module III outputs feature F03, and upsampling module IV outputs feature F04. The upsampling modules are used to unify the feature scale of the global processing module with the output feature scale of the hierarchical feature constructor.
[0073] In this embodiment, the segmentation head includes a splicing module III, a convolution module X, a convolution module XI, a batch normalization module VIII, and a ReLU6 activation function module VI; wherein the convolution kernels of convolution modules X and XI are both 1×1.
[0074] The input of stitching module III serves as the input of the segmentation head. The output of stitching module III is connected to the input of convolution module X. The output of convolution module X is connected to the input of ReLU6 activation function module VI. The output of ReLU6 activation function module VI is connected to the input of convolution module XI. The output of convolution module XI serves as the output of the segmentation head, outputting the image segmentation result. In practical applications, stitching module III has many input features. If the scales of the various features in the spatial dimensions (i.e., height and width) are inconsistent, the corresponding input features are also equipped with an upsampling module. That is, the upsampling module unifies the spatial scale of each input feature to the spatial scale of the sample image. The channel dimensions of the input features can be different, and stitching is achieved in the channel dimension.
[0075] In this embodiment, the adaptive feature fusion module includes a downsampling module, a concatenation module IV, a convolution module XII, a ReLU activation function module, a convolution module XIII, a softmax module II, and a concatenation module V; wherein, the convolution kernel size of convolution module XII is 3×3, and the convolution kernel size of convolution module XIII is 1×1;
[0076] The downsampling module performs downsampling operations on features F1, F2, F3 and F4 respectively to form features F1', F2', F3' and F4';
[0077] Features F1', F2', F3', and F4' are input into the stitching module IV. The output features of stitching module IV are input into convolution module XII. The output features of convolution module XII are input into the ReLU activation function module. The output features of the ReLU activation function module are input into convolution module XIII. The output features of convolution module XIII are input into softmax module II. Softmax module II outputs feature weights, which correspond one-to-one with features F1', F2', F3', and F4'. Features F1', F2', F3', and F4' are multiplied by their corresponding feature weights and then input into stitching module V. The output features of stitching module V serve as the output of the adaptive feature fusion module. Through this structure, dynamic integration of multi-scale features can be achieved through a spatial context-aware mechanism, preserving the integrity of the original features to the greatest extent.
[0078] The depthwise convolutional modules mentioned above are all existing depthwise separable convolutional modules. (See the attached diagram.) This indicates element-wise addition. Mul represents element-wise multiplication, and Mul represents the multiplication operation.
[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. A method for image segmentation of diseased areas on grape leaves using both frequency and spatial domains, characterized in that: Includes the following steps: S1. Acquire grape leaf sample image information and preprocess the grape leaf sample image information; S2. Construct an image segmentation model based on the co-domain and spatial domain. The image segmentation model includes an input convolution module, a hierarchical feature builder for multi-level feature extraction, an adaptive feature fusion module, a global processing module, and a segmentation head. The input convolution module is used to perform preliminary feature extraction processing on the input image. The output features of the input convolution module are input into the hierarchical feature builder. The hierarchical feature builder outputs features at four different scales, namely F1, F2, F3 and F4. Features F1, F2, and F3 are all input into the adaptive feature fusion module. The output features of the adaptive feature fusion module are input into the global processing module. The global processing module outputs four features, namely FO1, FO2, FO3, and FO4. Feature FO1 is added element-wise with feature F2 and then input into the segmentation head. Feature FO2 is added element-wise with feature F3 and then input into the segmentation head. Feature FO3 is added element-wise with feature F4 and then input into the segmentation head. Feature F4 is also input into the segmentation head. Feature F4 is also input into the average pooling module. The output features of the average pooling module are input into the segmentation head. The segmentation head outputs the segmentation result. The preprocessed grape leaf sample image information is input into the image segmentation model for training; S3. Obtain the real-time image information of the grape leaves to be segmented, and input the preprocessed real-time image information of the grape leaves into the trained image segmentation model to obtain the segmentation result.
2. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 1, characterized in that: The hierarchical feature constructor includes a first-stage processing module, a second-extreme processing module, a third-stage processing module, and a fourth-stage processing module. The first-stage processing module includes two cascaded gated convolutional units, and the first-stage processing module outputs feature F1; the second-stage processing module includes three cascaded gated convolutional units, and the second-stage processing module outputs feature F2; the third-stage processing module includes three cascaded multi-scale perception modules, and the third-stage processing module outputs feature F3; the fourth-stage processing module includes two multi-scale perception modules, and the fourth-stage processing module outputs feature F4; the gated convolutional unit structures of the first-stage and second-stage processing modules are the same; the multi-scale perception module structures of the third-stage and fourth-stage processing modules are the same.
3. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 2, characterized in that: The gated convolutional unit includes a convolution module I, a segmentation module I, a depthwise convolution module I, a batch normalization module I, a ReLU6 activation function module I, a splicing module I, a convolution module II, a batch normalization module II, and a GELU activation function module I; The input of convolution module I serves as the input of the gated convolutional unit. The output features of convolution module I are input to segmentation module I, which divides the input features into three features along the channel direction: feature GF1, feature GF2, and feature GF3. Feature GF2 is input to depth convolution module I, and the output features of depth convolution module I are input to batch normalization module I. The output features of batch normalization module I are input to ReLU6 activation function module I, and the output features of ReLU6 activation function module I are input to concatenation module I. Concatenation module I concatenates feature GF1 and the output features of ReLU6 activation function module I. Feature GF3 is input to GELU activation function module I. The output features of GELU activation function module I are multiplied element-wise with the output features of concatenation module I and then input to convolution module II. The output features of convolution module II are input to batch normalization module II, and the output features of batch normalization module II are added element-wise with the input features of convolution module I to serve as the output features of the gated convolutional unit.
4. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 2, characterized in that: The multi-scale perception module includes convolution module III, segmentation module II, depthwise convolution module II, depthwise convolution module III, depthwise convolution module IV, batch normalization module III, batch normalization module IV, batch normalization module V, batch normalization module VI, batch normalization module VII, ReLU6 activation function module II, ReLU6 activation function module III, ReLU6 activation function module IV, ReLU6 activation function module V, concatenation module II, convolution module IV, convolution module V, convolution module VI, and sigmoid function module; The input of convolution module III serves as the input of the multi-scale perception module. The output features of convolution module III are input to segmentation module II. Segmentation module II divides the input features into three parts along the channel direction: feature MF1, feature MF2, and feature MF3. Feature MF1 is input to depth convolution module II. The output features of depth convolution module II are input to batch normalization module III. The output features of batch normalization module III are input to ReLU6 activation function module II. Feature MF2 is input into deep convolution module III, the output feature of deep convolution module III is input into batch normalization module IV, and the output feature of batch normalization module IV is input into ReLU6 activation function module III; feature MF3 is input into deep convolution module IV, the output feature of deep convolution module IV is input into batch normalization module V, and the output feature of batch normalization module V is input into ReLU6 activation function module IV; wherein the dilation coefficients of deep convolution module II, deep convolution module III, and deep convolution module IV are 1, 3, and 5, respectively; The output features of RELU6 activation function modules II, III, and IV are all input into concatenation module II. The output features of concatenation module II are input into convolution module IV. The input and output features of convolution module IV are input into batch normalization module VI. The output features of batch normalization module VI are input into RELU6 activation function module V. The output features of RELU6 activation function module V are input into convolution module V. The output features of convolution module V are input into batch normalization module VII. The output features of batch normalization module VII are input into sigmoid function module. The output features of sigmoid function module are multiplied element-wise with the output features of concatenation module II and then input into convolution module VI. The output features of convolution module VI are added element-wise with the input features of convolution module III to obtain the final output result of the multi-scale perception module.
5. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 1, characterized in that: The global processing module includes a global processing module I and a global processing module II with identical structures and connected in series; the global processing module I adopts a no-offset window mode, and the global processing module II adopts a sliding window mode. The global processing module I includes a window partitioning module, a frequency domain spatial collaborative attention module, a window restoration module, a convolution module VIII, a convolution module IX, a depthwise convolution module V, a convolution module attention module, and a feedforward neural network layer; The input features of global processing module I are input into the window partitioning module and convolution module VIII, respectively. The output features of the window partitioning module are input into the frequency domain spatial collaborative attention module. The output features of the frequency domain spatial collaborative attention module are input into the window restoration module. The output features of convolution module VIII are input into the deep convolution module V. The output features of deep convolution module V are input into the convolution module attention module. The output features of the convolution module attention module are input into convolution module IX. The output features of convolution module IX and window restoration module are summed element-wise and then input into the feedforward neural network layer. The output of the feedforward neural network layer serves as the output of global processing module I.
6. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 5, characterized in that: The frequency-domain and spatial-domain collaborative attention module includes linear layer I, linear layer II, linear layer III, Fourier transform module, basic frequency-domain filter, global average pooling layer, Act function module, inverse Fourier transform module, and softmax module I; The input window partitioning module of the linear layer I, linear layer II and linear layer III outputs the features, with linear layer I outputting the value feature V, linear layer II outputting the query feature Q, and linear layer III outputting the key feature K; The value feature V is input to the Fourier transform module and the global average pooling layer. The global average pooling layer outputs modulation parameters. The value feature V is also input to the base filter. The modulation parameters are input to the base filter. The output feature of the base filter is multiplied element-wise with the output feature of the Fourier transform module and then input to the Act function module. The output feature of the Act function module is input to the inverse Fourier transform module. The query feature Q and the key feature K are multiplied and then input to the softmax module I. The output feature of the softmax module I is multiplied with the output feature of the inverse Fourier transform module and then used as the output feature of the frequency-domain spatial-domain collaborative attention module.
7. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 5, characterized in that: The output of the global processing module is further provided with upsampling module I, upsampling module II, upsampling module III and upsampling module IV; all upsampling modules I, II, III and IV input the same feature, and upsampling module I outputs feature F01, upsampling module II outputs feature F02, upsampling module III outputs feature F03 and upsampling module IV outputs feature F04.
8. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 1, characterized in that: The segmentation head includes a splicing module III, a convolution module X, a convolution module XI, a batch normalization module VIII, and a ReLU6 activation function module VI; The input of the stitching module III serves as the input of the segmentation head. The output of the stitching module III is connected to the input of the convolution module X. The output of the convolution module X is connected to the input of the ReLU6 activation function module VI. The output of the ReLU6 activation function module VI is connected to the input of the convolution module XI. The output of the convolution module XI serves as the output of the segmentation head, outputting the image segmentation result.
9. The method for image segmentation of grape leaf disease areas in the coordinated frequency and spatial domains according to claim 1, characterized in that: The adaptive feature fusion module includes a downsampling module, a concatenation module IV, a convolution module XII, a ReLU activation function module, a convolution module XIII, a softmax module II, and a concatenation module V; The downsampling module performs downsampling operations on features F1, F2, F3 and F4 respectively to form features F1', F2', F3' and F4'; Features F1', F2', F3', and F4' are input into the stitching module IV. The output features of stitching module IV are input into convolution module XII. The output features of convolution module XII are input into the ReLU activation function module. The output features of the ReLU activation function module are input into convolution module XIII. The output features of convolution module XIII are input into softmax module II. Softmax module II outputs feature weights, which correspond one-to-one with features F1', F2', F3', and F4'. Features F1', F2', F3', and F4' are multiplied by their corresponding feature weights and then input into stitching module V. The output features of stitching module V serve as the output of the adaptive feature fusion module.