Image segmentation network based on state memory calibration and plant leaf disease image segmentation method
Patent Information
- Application Number
- CN202610920556.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-24
- Publication Date
- 2026-08-07
AI Technical Summary
[0003]对于经济作物的病虫害的情况预测一般都是基于作物叶片的病害区域来进行分析判断，对于作物叶片的病虫害区域检测一般都是基于神经网络来实现图像分割，现有技术中，对于作物叶片的病害图像分割一般采用神经网络，比如CNN网络以及Transformer网络为代表的神经网络，现有的网络主要通过注意力增强、多尺度特征提取和上下文信息聚合等方式不断提升病害区域的表征能力，从而来提升分割精度，但是，在作物叶片病害图像分割中，对于全局信息利用不充分，全局信息的利用更多体现在特征融合与增强上，难以在特征传播过程中持续传递病害相关信息，也就是说：由于作物叶片病害分割不仅涉及病斑区域的整体语义建模，还面临病斑尺度变化、局部结构复杂以及复杂背景干扰等问题，而且病斑区域通常呈现出分布离散、尺度差异明显和形态不规则等特点，那么现有的分割网络难以同时兼顾局部细节刻画与整体语义表达，而且复杂的背景干扰容易影响病害现骨干特征的稳定表达，从而降低病害区域的判别准确性
[0031]本发明的有益效果：通过本发明，通过构建多层级的状态记忆校准网络，该网络以空间描述器和序列学习器构成非对称解耦构架，由空间描述器侧重提取局部空间信息，序列学习期侧重跨区域语义关系建模，从而能够有效兼顾局部细节刻画以及整体语义表达，进而有效确保最终作物病害区域图像分割结果的精度，为后续作物的病虫害防治提供准确的数据支持。
Smart Images
Figure CN122530592A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an image segmentation network and a segmentation method, and more particularly to an image segmentation network based on state memory calibration and a method for segmenting plant leaf disease images. Background Technology
[0002] In cash crops, it is often necessary to predict crop diseases and pests and formulate corresponding prevention measures; otherwise, the yield of cash crops will be seriously affected.
[0003] Predicting the status of pests and diseases in economic crops generally relies on analyzing and judging the diseased areas on crop leaves. Detecting these diseased areas typically involves image segmentation using neural networks. Current technologies for segmenting crop leaf disease images generally employ neural networks, such as CNNs and Transformers. These networks primarily enhance the representation of diseased areas through attention enhancement, multi-scale feature extraction, and contextual information aggregation, thereby improving segmentation accuracy. However, in crop leaf disease image segmentation, global information is not fully utilized. Its utilization is primarily focused on feature fusion and enhancement, making it difficult to continuously transmit disease-related information during feature propagation. In other words, crop leaf disease segmentation involves not only overall semantic modeling of lesion areas but also challenges such as lesion scale variations, complex local structures, and complex background interference. Furthermore, lesion areas often exhibit discrete distribution, significant scale differences, and irregular shapes. Therefore, existing segmentation networks struggle to simultaneously capture local details and express overall semantics. Complex background interference can also affect the stable expression of the disease's core features, thus reducing the accuracy of disease area discrimination.
[0004] Therefore, in order to solve the above-mentioned technical problems, it is urgent to propose a new technical approach. Summary of the Invention
[0005] In view of this, the purpose of this invention is to provide an image segmentation network based on state memory calibration and a method for segmenting plant leaf disease images. By constructing a multi-level state memory calibration network, which uses a spatial descriptor and a sequence learner to form an asymmetric decoupled architecture, the spatial descriptor focuses on extracting local spatial information, while the sequence learner focuses on modeling cross-regional semantic relationships. This effectively balances the depiction of local details and the overall semantic expression, thereby effectively ensuring the accuracy of the final crop disease area image segmentation results and providing accurate data support for subsequent crop pest and disease control.
[0006] This invention provides an image segmentation network based on state memory calibration, comprising a primary feature extraction module, four structurally identical state memory calibration networks, and a segmentation head;
[0007] The input features of the primary feature extraction module are input into four sequentially connected state memory calibration networks. The output features of the last state calibration memory network are input into the segmentation head, and the segmentation head outputs the segmentation result.
[0008] The state memory calibration network includes a spatial descriptor for extracting local spatial information, a sequence learner for extracting sequence information for cross-regional semantic modeling, a recovery module, and an input adapter.
[0009] The input terminal of the input adapter serves as the input terminal of the state memory calibration network. The output features of the input adapter are respectively input into the spatial descriptor and the sequence learner. The input features of the spatial descriptor and the sequence learner are input into the recovery module for feature fusion. The output terminal of the recovery module serves as the output terminal of the state memory calibration network.
[0010] Furthermore, the spatial descriptor includes a 1×1 convolution module I, a 7×7 depthwise convolution module, a 3×3 depthwise convolution module I, a SiLU activation function module I, and a 1×1 convolution module II;
[0011] The input of the 1×1 convolution module I serves as the input of the spatial descriptor. The output features of the 1×1 convolution module I are input into the 7×7 depth convolution module, the output features of the 7×7 depth convolution module are input into the 3×3 depth convolution module I, the output features of the 3×3 depth convolution module I are input into the SiLU activation function module I, and the output features of the SiLU activation function module I are input into the 1×1 convolution module II. The output features of the 1×1 convolution module II are added element-wise to the input features of the 1×1 convolution module I to serve as the output features of the spatial descriptor.
[0012] Furthermore, the sequence learner includes a layer normalization module I, a layer normalization module II, a 1×1 convolution module III, a 1×1 convolution module IV, a 1×1 convolution module V, a 1D deep convolution module, a SiLU activation function module II, a SiLU activation function module III, a 2D selective scanning module, a feature fusion module, a state memory generation module, a state memory module, a state calibrator, a wavelet-enhanced feedforward network, and a recovery module;
[0013] The layer normalization module I receives forward and reverse feature sequences. The output features of layer normalization module I are input to 1×1 convolution modules III and IV. The output features of 1×1 convolution module III are input to a 1D depthwise convolution module. The output features of the 1D depthwise convolution module are input to SiLU activation function module II. The output features of SiLU activation function module II are input to a 2D selective scan module. The output features of the 2D selective scan module are input to layer normalization module II. The output features of 1×1 convolution module IV are input to SiLU activation function module III. The output features of block III are multiplied element-wise with the output features of layer normalization module II and then input into 1×1 convolution module V. The output features of 1×1 convolution module V are added element-wise with the input features of layer normalization module I and then input into feature fusion module. The output features of feature fusion module are input into state calibrator. The output features of state calibrator are input into wavelet enhancement feedforward network. The output features of wavelet enhancement feedforward network are input into recovery module. Recovery module also receives output features of spatial descriptor. Recovery module is used to fuse output features of wavelet enhancement feedforward network and output features of spatial descriptor and then input them into segmentation head.
[0014] The two-dimensional selective scanning module is used to process the forward and reverse sequences and then output forward hidden state features and reverse hidden state features to the state memory generation module. The output features of the state memory generation module are input to the state memory module. The state memory generation module also receives the forward scanning features and reverse scanning features output by the two-dimensional selective scanning module. The output features of the state memory module are input to the state calibrator.
[0015] Furthermore, the state memory generation module includes a global average pooling module, a multilayer perceptron module I, a softmax module, a linear layer, a layer normalization module III, and a SiLU activation function module IV;
[0016] The forward and reverse scanning features are added element-wise and then input into the global average pooling module. The output features of the global average pooling module are input into the multilayer perceptron module I. The output features of the multilayer perceptron module I are input into the softmax module. The softmax module outputs the forward hidden state weights and the reverse hidden state weights. The feature obtained by multiplying the forward hidden state weights and the forward hidden state features with the feature obtained by multiplying the reverse hidden state weights and the reverse hidden state features is added element-wise and then input into the linear layer. The output features of the linear layer are input into the layer normalization module III. The output features of the layer normalization module III are input into the SiLU activation function module IV. The output of the SiLU activation function module IV serves as the output of the state memory generation module, outputting features to the state memory module.
[0017] Furthermore, the state calibrator includes a multilayer perceptron module II, a stitching module, a SiLU activation function module V, and a subtraction module;
[0018] The output features of the state memory module are input to the multilayer perceptron module II. The multilayer perceptron module II outputs scaling term features and offset term features. The scaling term features are multiplied element-wise with the output features of the feature fusion module and then added element-wise with the offset term features before being input to the stitching module. The stitching module also receives the output features of the feature fusion module. The output features of the stitching module are input to the SiLU activation function module V.
[0019] The subtraction module receives the output features of the feature fusion module and the scaling term features. The scaling term features are multiplied element-wise by the output features of the feature fusion module and then added element-wise by the offset term features to form the output features of the state calibrator. The output features of the subtraction module are multiplied element-wise by the output features of the SiLU activation function module V and then added element-wise by the output features of the feature fusion module to form the output features of the state calibrator.
[0020] Furthermore, the wavelet-enhanced feedforward network includes a reshaping module I, a 1×1 convolution module VI, a 3×3 depth convolution module II, a segmentation module, a SiLU activation function module V, a 1×1 convolution module VII, a graphics block partitioning module, a discrete wavelet transform module, a discrete wavelet inverse transform module, a block merging module, and a reshaping module II;
[0021] The output features of the input state calibrator are input into the reshaping module I. The output features of the reshaping module I are input into the 1×1 convolution module VI. The output features of the 1×1 convolution module VI are input into the 3×3 depth convolution module II. The output features of the 3×3 depth convolution module II are input into the segmentation module. The output features of the segmentation module are input into the SiLU activation function module V. The output features of the SiLU activation function module V and the output features of the segmentation module are multiplied element-wise and then input into the 1×1 convolution module VII. The output features of the 1×1 convolution module VII are input into the graphic block partitioning module. The output features of the graphic block partitioning module are input into the discrete wavelet transform module. The output features of the discrete wavelet transform module are multiplied with the learnable weight factor W and then input into the discrete wavelet inverse transform module. The output features of the discrete wavelet inverse transform module are input into the block merging module. The output features of the block merging module are input into the reshaping module II. The output features of the reshaping module II are used as the output features of the wavelet enhancement feedforward network.
[0022] Furthermore, the recovery module includes an adaptive upsampling module, which takes the output features of the wavelet-enhanced feedforward network as input, multiplies the output features of the spatial descriptor with the learnable parameter γ, and then adds them element-wise to the output features of the adaptive upsampling module to form the output features of the recovery module.
[0023] Furthermore, the input adapter includes a 3×3 depthwise convolution module III and a SiLU activation function module VI;
[0024] The input of the 3×3 depth convolution module III serves as the input of the input adapter, and the output of the 3×3 depth convolution module III is connected to the input of the SiLU activation function module VI. The output of the SiLU activation function module VI serves as the output of the input adapter.
[0025] Furthermore, the primary feature extraction module includes two feature extractors with identical structures connected in series;
[0026] The feature extractor includes a 3×3 depth convolution module IV and a SiLU activation function module VII; the input of the 3×3 depth convolution module IV serves as the input of the input adapter, the output of the 3×3 depth convolution module IV is connected to the input of the SiLU activation function module VII, and the output of the SiLU activation function module VII serves as the output of the input adapter.
[0027] Accordingly, the present invention also provides a method for segmenting plant leaf disease images based on the above-mentioned image segmentation network, comprising the following steps:
[0028] S1. Acquire sample images of plant leaves and preprocess the plant leaf sample images;
[0029] S2. Construct an image segmentation network based on state memory calibration, and input the preprocessed plant leaf sample images into the image segmentation network for training;
[0030] S3. Acquire real-time images of the leaves of the plant to be tested, preprocess the real-time images, and input them into the trained image segmentation network to obtain the image segmentation results of the diseased parts.
[0031] The beneficial effects of this invention are as follows: By constructing a multi-level state memory calibration network, which uses a spatial descriptor and a sequence learner to form an asymmetric decoupled architecture, the spatial descriptor focuses on extracting local spatial information, while the sequence learner focuses on modeling cross-regional semantic relationships. This effectively balances the depiction of local details with the overall semantic expression, thereby ensuring the accuracy of the final crop disease area image segmentation results and providing accurate data support for subsequent crop pest and disease control. Attached Figure Description
[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments:
[0033] Figure 1 This is a schematic diagram of the structure of the present invention.
[0034] Figure 2 This is a schematic diagram of the spatial descriptor structure of the present invention.
[0035] Figure 3 This is a schematic diagram of the sequence learner structure of the present invention.
[0036] Figure 4 This is a schematic diagram of the state memory generation module of the present invention.
[0037] Figure 5 This is a schematic diagram of the state calibrator structure of the present invention.
[0038] Figure 6 This is a schematic diagram of the wavelet-enhanced feedforward network structure of the present invention.
[0039] Figure 7 This is a schematic diagram of the recovery module structure of the present invention.
[0040] Figure 8 This is a schematic diagram of the input adapter structure of the present invention.
[0041] Figure 9 This is a schematic diagram of the primary feature extraction module of the present invention.
[0042] Figure 10 This is a schematic diagram of the process for the plant leaf disease image segmentation method of the present invention. Detailed Implementation
[0043] The present invention will be further described in detail below:
[0044] This invention provides an image segmentation network based on state memory calibration, comprising a primary feature extraction module (represented as Stem in the figures), four structurally identical state memory calibration networks, and a segmentation head;
[0045] The input features of the primary feature extraction module are input into four sequentially connected state memory calibration networks. The output features of the last state calibration memory network are input into the segmentation head, and the segmentation head outputs the segmentation result.
[0046] The state memory calibration network includes a spatial descriptor (denoted by SD in the figure) for extracting local spatial information, a sequence learner (denoted by SL in the figure) for extracting sequence information for cross-regional semantic modeling, a recovery module (denoted by Restore in the figure), and an input adapter (denoted by SIA in the figure).
[0047] The input terminal of the input adapter serves as the input terminal of the state memory calibration network. The output features of the input adapter are respectively input into the spatial descriptor and the sequence learner. The input features of the spatial descriptor and the sequence learner are input into the recovery module for feature fusion. The output terminal of the recovery module serves as the output terminal of the state memory calibration network. Through this invention, a multi-level state memory calibration network is constructed. This network uses a spatial descriptor and a sequence learner to form an asymmetric decoupled architecture. The spatial descriptor focuses on extracting local spatial information, while the sequence learner focuses on modeling cross-regional semantic relationships. This effectively balances local detail depiction and overall semantic expression, thereby effectively ensuring the accuracy of the final crop disease area image segmentation results and providing accurate data support for subsequent crop pest and disease control.
[0048] Wherein: the primary feature extraction module includes two feature extractors with identical structures connected in series;
[0049] The feature extractor includes a 3×3 depthwise convolutional module IV and a SiLU activation function module VII. The input of the 3×3 depthwise convolutional module IV serves as the input of an input adapter, and its output is connected to the input of the SiLU activation function module VII. The output of the SiLU activation function module VII serves as the output of the input adapter. The primary feature extraction module performs initial downsampling and shallow feature extraction on the input image, providing basic features for subsequent processing.
[0050] Each state memory calibration network has the same structure, but the sampling stride is different for each network. That is, the stride of the convolutional modules in each network is different. In the first state memory calibration network, the stride of both the spatial descriptor and the sequence learner is 1; in the second state memory calibration network, the stride of both the spatial descriptor and the sequence learner is 2; in the third state memory calibration network, the stride of both the spatial descriptor and the sequence learner is 1 and 2 respectively; and in the fourth state memory calibration network, the stride of both the spatial descriptor and the sequence learner is 1 and 4 respectively.
[0051] In the first and second state memory calibration networks, the spatial descriptor and sequence learner have the same step size, which is used to gradually build the basic representation of local structure and regional semantics. The third and fourth state memory calibration networks use different step sizes, which makes the spatial descriptor have high spatial resolution, while the input adapters of the third and fourth state memory calibration networks continue to preserve lesion boundaries and texture details. It also enables the sequence learner to model the semantic dependencies between lesion regions and between lesions and the background in a larger receptive field.
[0052] Among them, the first to fourth are determined by the input-to-output direction, and the last state memory calibration network mentioned above refers to the fourth state memory calibration network.
[0053] The segmentation head adopts an existing segmentation head, which generally consists of two cascaded 1×1 convolutional modules to form a semantic segmentation head.
[0054] In this embodiment, the spatial descriptor includes a 1×1 convolution module I, a 7×7 depthwise convolution module, a 3×3 depthwise convolution module I, a SiLU activation function module I, and a 1×1 convolution module II;
[0055] The input of the 1×1 convolution module I serves as the input of the spatial descriptor. The output features of the 1×1 convolution module I are input into the 7×7 depth convolution module, the output features of the 7×7 depth convolution module are input into the 3×3 depth convolution module I, the output features of the 3×3 depth convolution module I are input into the SiLU activation function module I, and the output features of the SiLU activation function module I are input into the 1×1 convolution module II. The output features of the 1×1 convolution module II are added element-wise to the input features of the 1×1 convolution module I to serve as the output features of the spatial descriptor. The spatial descriptor with the above structure uses 1×1 convolutional modules I and II to adjust the feature channels, compress redundant response information, and aggregate effective information; a 7×7 depth convolutional module extracts structural information of the lesion region within a larger local area; a 3×3 depth convolutional module I further supplements the local texture and detail response; and a SiLU activation function module I is used to introduce nonlinear features to enhance feature expression. Through the above structure, local spatial information of the lesion region can be effectively obtained, providing accurate data support for the final accurate segmentation of the disease (or lesion) image.
[0056] The depthwise convolutional modules mentioned above and below are both depthwise separable convolutional modules, which are existing technologies and will not be described in detail here.
[0057] In this embodiment, the sequence learner includes a layer normalization module I, a layer normalization module II, a 1×1 convolution module III, a 1×1 convolution module IV, a 1×1 convolution module V, a 1D deep convolution module (represented as DWconv1D in the figure), a SiLU activation function module II, a SiLU activation function module III, a 2D selective scanning module, a feature fusion module, a state memory generation module, a state memory module, a state calibrator, a wavelet-enhanced feedforward network, and a recovery module;
[0058] The layer normalization module I receives forward and reverse feature sequences. The output features of layer normalization module I are input to 1×1 convolution modules III and IV. The output features of 1×1 convolution module III are input to a 1D depthwise convolution module. The output features of the 1D depthwise convolution module are input to SiLU activation function module II. The output features of SiLU activation function module II are input to a 2D selective scan module. The output features of the 2D selective scan module are input to layer normalization module II. The output features of 1×1 convolution module IV are input to SiLU activation function module III. The output features of block III are multiplied element-wise with the output features of layer normalization module II and then input into 1×1 convolution module V. The output features of 1×1 convolution module V are added element-wise with the input features of layer normalization module I and then input into feature fusion module. The output features of feature fusion module are input into state calibrator. The output features of state calibrator are input into wavelet enhancement feedforward network. The output features of wavelet enhancement feedforward network are input into recovery module. Recovery module also receives output features of spatial descriptor. Recovery module is used to fuse output features of wavelet enhancement feedforward network and output features of spatial descriptor and then input them into segmentation head.
[0059] The two-dimensional selective scan module processes the forward and reverse sequences and outputs forward and reverse hidden state features to the state memory generation module. The output features of the state memory generation module are input to the state memory module. The state memory generation module also receives the forward and reverse scan features output by the two-dimensional selective scan module. The output features of the state memory module are input to the state calibrator. The two-dimensional selective scan module (i.e., 2D Selective Scan Module, abbreviated as 2DSSM, denoted by abbreviation in the figure) is a prior art technology. After processing the forward and reverse sequences, it outputs forward and reverse scan features to the layer normalization module II and the state memory generation module. At the same time, it also generates forward hidden state features (corresponding to the forward sequence) and reverse hidden state features (corresponding to the reverse sequence) and inputs them to the state memory generation module.
[0060] It should also be noted that the two-dimensional selective scanning module is not performed simultaneously, but rather in a time-division multiplexing manner. The feature fusion module is used to fuse the forward scanning features and the reverse scanning features.
[0061] The forward and reverse sequences are generated by the serialization embedding module. That is, the output features of the input adapter are input to the serialization embedding module for processing to generate the forward and reverse sequences. The reverse sequence is formed by reversing the forward sequence. This is existing technology and will not be described in detail here.
[0062] The state memory module is existing technology and will not be described in detail here.
[0063] Specifically, the state memory generation module includes a global average pooling module, a multilayer perceptron module I, a softmax module, a linear layer, a layer normalization module III, and a SiLU activation function module IV;
[0064] The forward and reverse scanning features (represented by yf and yb in the accompanying drawings, respectively) are element-wise summed and then input into the global average pooling module. The output features of the global average pooling module are input into the multilayer perceptron module I. The output features of the multilayer perceptron module I are input into the softmax module. The softmax module outputs the forward and reverse hidden state weights. The feature obtained by multiplying the forward hidden state weights with the forward hidden state features (represented by Hf in the accompanying drawings) and then multiplying the feature obtained by multiplying the reverse hidden state weights with the reverse hidden state features (represented by Hb in the accompanying drawings) element-wise is added and then input into the linear layer. The output features of the linear layer are input into the layer normalization module III. The output features of the layer normalization module III are input into the SiLU activation function module IV. The output of the SiLU activation function module IV serves as the output of the state memory generation module, outputting features to the state memory module. Through the above, state memory is constructed using dual hidden state features and adaptively fused, which can accurately reflect state information.
[0065] The state calibrator includes a multilayer perceptron module II, a stitching module, a SiLU activation function module V, and a subtraction module;
[0066] The output features of the state memory module (represented by FS in the attached figure) are input to the multilayer perceptron module II. The multilayer perceptron module II outputs scaling features (represented by scale) and offset features (represented by shift). The scaling features are multiplied element-wise with the output features of the feature fusion module and then added element-wise with the offset features before being input to the stitching module. The stitching module also receives the output features of the feature fusion module. The output features of the stitching module are input to the SiLU activation function module V.
[0067] The subtraction module receives the output features (represented by FY in the attached diagram) from the feature fusion module, and the feature formed by multiplying the scaling term features element-wise with the output features of the feature fusion module, and then adding the offset term features element-wise. The output features of the subtraction module are then multiplied element-wise with the output features of the SiLU activation function module V, and then added element-wise with the output features of the feature fusion module to form the output features of the state calibrator. Through this process, the original expression of the features can be effectively ensured, preventing feature weakening during feature transfer. While enhancing the lesion-related response, irrelevant interference is suppressed, thereby enhancing the stability and discriminative power of disease-related information expression in complex backgrounds.
[0068] Among them, the feature fusion module and the multilayer perceptron module are existing technologies, and their structures will not be described in detail here.
[0069] Sequence learners can accurately model cross-regional semantic dependencies between lesion areas and between lesions and the leaf background, enhancing detailed customer and overall semantic expression, and providing accurate data support for subsequent image segmentation.
[0070] In this embodiment, the wavelet enhancement feedforward network includes a reshaping module I, a 1×1 convolution module VI, a 3×3 depth convolution module II, a segmentation module, a SiLU activation function module V, a 1×1 convolution module VII, a patch partitioning module, a discrete wavelet transform module, a discrete wavelet inverse transform module, a patch merging module, and a reshaping module II;
[0071] The reshaping module I inputs the output features of the state calibrator. The output features of reshaping module I are then input into a 1×1 convolution module VI. The output features of 1×1 convolution module VI are input into a 3×3 depthwise convolution module II. The output features of 3×3 depthwise convolution module II are input into a segmentation module. The output features of the segmentation module are then input into a [missing information - likely a specific module or module]. The output features of SiLU activation function module V are element-wise multiplied with the output features of the segmentation module and then input into a 1×1 convolution module VII. The output features of 1×1 convolution module VII are then input into a graphics block partitioning module. (This graphics block partitioning module is existing technology; its English name is Patch.) The output features of the partition are input into the discrete wavelet transform module. The output features of the discrete wavelet transform module are multiplied by a learnable weight factor W (which can be learned and adjusted during network training to adaptively adjust the response intensity of different frequency components) and then input into the inverse discrete wavelet transform module. The output features of the inverse discrete wavelet transform module are input into the block merging module. The output features of the block merging module are input into the reshaping module II. The output features of the reshaping module II are used as the output features of the wavelet-enhanced feedforward network. After state space modeling and state memory calibration, the features need to be reshaped and refined. The discrimination of lesion regions depends on cross-regional semantic relationships and is also closely related to local structure and texture details. Through the above structure, the local structure and texture details in the calibrated features are specifically refined and reshaped. After state memory calibration, the reshaping and refinement of the feedforward features are completed, supplementing the local texture and detail representation.
[0072] In this embodiment, the recovery module includes an adaptive upsampling module. The adaptive upsampling module takes the output features of the wavelet-enhanced feedforward network (represented by FSL) as input, multiplies the output features of the spatial descriptor (represented by FSD) with the learnable parameter γ, and then adds them element-wise to the output features of the adaptive upsampling module to form the output features of the recovery module.
[0073] The input adapter includes a 3×3 depthwise convolution module III and a SiLU activation function module VI;
[0074] The input of the 3×3 depth convolution module III serves as the input of the input adapter, and the output of the 3×3 depth convolution module III is connected to the input of the SiLU activation function module VI. The output of the SiLU activation function module VI serves as the output of the input adapter.
[0075] In the attached diagram, the "+" sign indicates element-wise addition, and the "×" sign indicates element-wise multiplication. Depthwise convolution modules are represented as DWconv, for example, 3×3 depthwise convolution module Ⅳ is represented as 3×3DWconvⅣ; convolution modules are represented as conv, for example, 1×1 convolution module Ⅵ is represented as 1×1convⅥ; SiLU activation function modules are represented as SiLU, for example, SiLU activation function module Ⅶ is represented as SiLUⅦ; discrete wavelet transform modules are represented as DWT, and inverse discrete wavelet transform modules are represented as IDWT; multilayer perceptron modules are represented as MLP; layer normalization modules are represented as LNorm, for example, layer normalization module Ⅱ is represented as LNormⅡ; and global average pooling modules are represented as GAP.
[0076] Accordingly, the present invention also provides a method for segmenting plant leaf disease images based on the above-mentioned image segmentation network, comprising the following steps:
[0077] S1. Acquire sample images of plant leaves and preprocess the plant leaf sample images;
[0078] S2. Construct an image segmentation network based on state memory calibration, and input the preprocessed plant leaf sample images into the image segmentation network for training;
[0079] S3. Acquire real-time images of the leaves of the plant to be tested, preprocess the real-time images, and input them into the trained image segmentation network to obtain the image segmentation results of the diseased parts.
[0080] Image preprocessing includes operations such as noise reduction and contrast enhancement, which are existing technologies and will not be elaborated here.
[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.
Claims
1. An image segmentation network based on state memory calibration, characterized in that: It includes a primary feature extraction module, four structurally identical state memory calibration networks, and a segmentation head; The input features of the primary feature extraction module are input into four sequentially connected state memory calibration networks. The output features of the last state calibration memory network are input into the segmentation head, and the segmentation head outputs the segmentation result. The state memory calibration network includes a spatial descriptor for extracting local spatial information, a sequence learner for extracting sequence information for cross-regional semantic modeling, a recovery module, and an input adapter. The input terminal of the input adapter serves as the input terminal of the state memory calibration network. The output features of the input adapter are respectively input into the spatial descriptor and the sequence learner. The input features of the spatial descriptor and the sequence learner are input into the recovery module for feature fusion. The output terminal of the recovery module serves as the output terminal of the state memory calibration network.
2. The image segmentation network based on state memory calibration according to claim 1, characterized in that: The spatial descriptor includes a 1×1 convolution module I, a 7×7 depthwise convolution module, a 3×3 depthwise convolution module I, a SiLU activation function module I, and a 1×1 convolution module II; The input of the 1×1 convolution module I serves as the input of the spatial descriptor. The output features of the 1×1 convolution module I are input into the 7×7 depth convolution module, the output features of the 7×7 depth convolution module are input into the 3×3 depth convolution module I, the output features of the 3×3 depth convolution module I are input into the SiLU activation function module I, and the output features of the SiLU activation function module I are input into the 1×1 convolution module II. The output features of the 1×1 convolution module II are added element-wise to the input features of the 1×1 convolution module I to serve as the output features of the spatial descriptor.
3. The image segmentation network based on state memory calibration according to claim 1, characterized in that: The sequence learner includes a layer normalization module I, a layer normalization module II, a 1×1 convolution module III, a 1×1 convolution module IV, a 1×1 convolution module V, a 1D deep convolution module, a SiLU activation function module II, a SiLU activation function module III, a 2D selective scanning module, a feature fusion module, a state memory generation module, a state memory module, a state calibrator, a wavelet-enhanced feedforward network, and a recovery module. The layer normalization module I receives forward and reverse feature sequences. The output features of layer normalization module I are input to 1×1 convolution modules III and IV. The output features of 1×1 convolution module III are input to a 1D depthwise convolution module. The output features of the 1D depthwise convolution module are input to SiLU activation function module II. The output features of SiLU activation function module II are input to a 2D selective scan module. The output features of the 2D selective scan module are input to layer normalization module II. The output features of 1×1 convolution module IV are input to SiLU activation function module III. The output features of block III are multiplied element-wise with the output features of layer normalization module II and then input into 1×1 convolution module V. The output features of 1×1 convolution module V are added element-wise with the input features of layer normalization module I and then input into feature fusion module. The output features of feature fusion module are input into state calibrator. The output features of state calibrator are input into wavelet enhancement feedforward network. The output features of wavelet enhancement feedforward network are input into recovery module. Recovery module also receives output features of spatial descriptor. Recovery module is used to fuse output features of wavelet enhancement feedforward network and output features of spatial descriptor and then input them into segmentation head. The two-dimensional selective scanning module is used to process the forward and reverse sequences and then output forward hidden state features and reverse hidden state features to the state memory generation module. The output features of the state memory generation module are input to the state memory module. The state memory generation module also receives the forward scanning features and reverse scanning features output by the two-dimensional selective scanning module. The output features of the state memory module are input to the state calibrator.
4. The image segmentation network based on state memory calibration according to claim 3, characterized in that: The state memory generation module includes a global average pooling module, a multilayer perceptron module I, a softmax module, a linear layer, a layer normalization module III, and a SiLU activation function module IV. The forward and reverse scanning features are added element-wise and then input into the global average pooling module. The output features of the global average pooling module are input into the multilayer perceptron module I. The output features of the multilayer perceptron module I are input into the softmax module. The softmax module outputs the forward hidden state weights and the reverse hidden state weights. The feature obtained by multiplying the forward hidden state weights and the forward hidden state features with the feature obtained by multiplying the reverse hidden state weights and the reverse hidden state features is added element-wise and then input into the linear layer. The output features of the linear layer are input into the layer normalization module III. The output features of the layer normalization module III are input into the SiLU activation function module IV. The output of the SiLU activation function module IV serves as the output of the state memory generation module, outputting features to the state memory module.
5. The image segmentation network based on state memory calibration according to claim 3, characterized in that: The state calibrator includes a multilayer perceptron module II, a stitching module, a SiLU activation function module V, and a subtraction module; The output features of the state memory module are input to the multilayer perceptron module II. The multilayer perceptron module II outputs scaling term features and offset term features. The scaling term features are multiplied element-wise with the output features of the feature fusion module and then added element-wise with the offset term features before being input to the stitching module. The stitching module also receives the output features of the feature fusion module. The output features of the stitching module are input to the SiLU activation function module V. The subtraction module receives the output features of the feature fusion module and the scaling term features. The scaling term features are multiplied element-wise by the output features of the feature fusion module and then added element-wise by the offset term features to form the output features of the state calibrator. The output features of the subtraction module are multiplied element-wise by the output features of the SiLU activation function module V and then added element-wise by the output features of the feature fusion module to form the output features of the state calibrator.
6. The image segmentation network based on state memory calibration according to claim 3, characterized in that: The wavelet enhancement feedforward network includes a reshaping module I, a 1×1 convolution module VI, a 3×3 depth convolution module II, a segmentation module, a SiLU activation function module V, a 1×1 convolution module VII, a graphic block partitioning module, a discrete wavelet transform module, a discrete wavelet inverse transform module, a block merging module, and a reshaping module II. The output features of the input state calibrator are input into the reshaping module I. The output features of the reshaping module I are input into the 1×1 convolution module VI. The output features of the 1×1 convolution module VI are input into the 3×3 depth convolution module II. The output features of the 3×3 depth convolution module II are input into the segmentation module. The output features of the segmentation module are input into the SiLU activation function module V. The output features of the SiLU activation function module V and the output features of the segmentation module are multiplied element-wise and then input into the 1×1 convolution module VII. The output features of the 1×1 convolution module VII are input into the graphic block partitioning module. The output features of the graphic block partitioning module are input into the discrete wavelet transform module. The output features of the discrete wavelet transform module are multiplied with the learnable weight factor W and then input into the discrete wavelet inverse transform module. The output features of the discrete wavelet inverse transform module are input into the block merging module. The output features of the block merging module are input into the reshaping module II. The output features of the reshaping module II are used as the output features of the wavelet enhancement feedforward network.
7. The image segmentation network based on state memory calibration according to claim 3, characterized in that: The recovery module includes an adaptive upsampling module. The adaptive upsampling module takes the output features of the wavelet-enhanced feedforward network as input. The output features of the spatial descriptor are multiplied by the learnable parameter γ and then added element-wise to the output features of the adaptive upsampling module to form the output features of the recovery module.
8. The image segmentation network based on state memory calibration according to claim 1, characterized in that: The input adapter includes a 3×3 depthwise convolution module III and a SiLU activation function module VI; The input of the 3×3 depth convolution module III serves as the input of the input adapter, and the output of the 3×3 depth convolution module III is connected to the input of the SiLU activation function module VI. The output of the SiLU activation function module VI serves as the output of the input adapter.
9. The image segmentation network based on state memory calibration according to claim 1, characterized in that: The primary feature extraction module includes two identical feature extractors connected in series. The feature extractor includes a 3×3 depth convolution module IV and a SiLU activation function module VII; the input of the 3×3 depth convolution module IV serves as the input of the input adapter, the output of the 3×3 depth convolution module IV is connected to the input of the SiLU activation function module VII, and the output of the SiLU activation function module VII serves as the output of the input adapter.
10. A method for segmenting plant leaf disease images based on the image segmentation network described in any one of claims 1-9, characterized in that: Includes the following steps: S1. Acquire sample images of plant leaves and preprocess the plant leaf sample images; S2. Construct an image segmentation network based on state memory calibration, and input the preprocessed plant leaf sample images into the image segmentation network for training; S3. Acquire real-time images of the leaves of the plant to be tested, preprocess the real-time images, and input them into the trained image segmentation network to obtain the image segmentation results of the diseased parts.