Mixed deep learning method and device for ordered severity grading of poplar black spot, and storage medium

Abstract

The invention discloses a mixed deep learning method and device for ordered severity grading of poplar black spot, and a storage medium, and belongs to the technical field of smart agriculture and computer vision. According to the method, a mixed deep learning model named PoplarFormer is constructed, and the model realizes bidirectional deep fusion of global context features (extracted by an EVA-02 visual Transform backbone network) and local fine-grained features (extracted by a parallel multi-scale convolution enhancement module) in multiple network stages through a collaborative feature interaction module; meanwhile, a CORAL ordered regression loss function is adopted for optimization in a training stage, so that the model can learn an internal logic sequence between severity levels. According to the method, through the collaborative design of'architecture-loss function ', the core pain points of insufficient feature fusion and neglect of grade orderliness in the prior art are solved, and automatic grading which is more accurate and higher in logic consistency compared with the prior art is realized.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and smart agriculture, and in particular to a hybrid deep learning method, device and storage medium for ordered severity grading of poplar black spot disease. Background Technology

[0002] As a fundamental component of the global food supply, plants are vulnerable to various diseases, and environmental factors often exacerbate these diseases, leading to severe yield losses. Plant diseases are a major challenge in plant cultivation and management, causing significant damage to yields and triggering serious economic consequences. Poplar black spot (caused by Marssonina brunnea) is one of the most destructive foliage diseases in the global poplar industry, resulting in severe economic losses. Currently, the assessment of the severity of this disease mainly relies on subjective visual inspection by forestry experts. This method is inefficient, inconsistent, and lacks unified quantitative standards, making it difficult to meet the needs of large-scale, precise forest health management.

[0003] In recent years, deep learning techniques, especially convolutional neural networks (CNNs) and visual transformers (ViTs), have been applied to plant disease identification. CNN models (such as LeafConvNeXt and DH-CNN) can automatically learn hierarchical features, but their inherent local receptive fields limit their ability to capture global contextual information of images. ViT models (such as SwinTransformer) effectively model long-range dependencies through self-attention mechanisms, but they are insufficient in handling subtle, localized lesion texture features. Although some studies have attempted to construct hybrid architectures of CNNs and ViTs (such as AgriTL-ViT and ViT-CoMer) to combine the advantages of both, there is still room for optimization in how to efficiently and synergistically achieve bidirectional complementary fusion of global and local features.

[0004] Furthermore, existing methods for assessing the severity of plant diseases often treat the problem as a simple classification task and use cross-entropy loss for optimization, ignoring the inherent orderliness between severity levels, resulting in poor logical consistency in model predictions.

[0005] Therefore, there is an urgent need in the existing technology for an automated system that can scientifically quantify and collaboratively integrate multi-scale features to classify severity in an orderly manner. Summary of the Invention

[0006] (a) Purpose of the invention

[0007] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a hybrid deep learning method, device, and storage medium for ordered severity grading of poplar black spot disease. This method aims to achieve objective, accurate, and automated quantitative assessment of disease severity, and ensure that the prediction results conform to the inherent logical sequence of disease development.

[0008] (II) Technical Solution

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] In a first aspect, the present invention provides a hybrid deep learning method for ordered severity grading of poplar black spot disease, comprising the following steps:

[0011] Step S1: Acquire an image of the poplar leaf to be tested, and preprocess the image;

[0012] Step S2: Input the preprocessed image into the trained PoplarFormer grading model, and the model outputs the severity level of poplar black spot disease;

[0013] The PoplarFormer hierarchical model is a hybrid neural network built on the EVA-02 visual Transformer backbone network. It extracts local fine-grained features of the image through a parallel multi-scale convolution enhancement module, and performs bidirectional fusion of the local fine-grained features with the global context features extracted by the EVA-02 backbone network through a collaborative feature interaction module. In this scheme, "parallel" refers to parallel branches in the structure, rather than necessarily parallel processing in computation.

[0014] Furthermore, the PoplarFormer grading model is optimized using the CORAL loss function during the training phase to learn the ordered relationship between severity levels.

[0015] Furthermore, the PoplarFormer hierarchical model is constructed and trained in the following manner:

[0016] Construct a dual-branch network, which includes the EVA-02 backbone network as the main branch and a convolutional branch as the parallel branch;

[0017] A multi-scale convolution enhancement module is set in the convolution branch to generate a multi-scale feature pyramid from the input image, and to extract local spatial features of multiple receptive fields in parallel using convolutional layers with different kernel sizes.

[0018] In the N consecutive Transformer module stages of the EVA-02 backbone network, a collaborative feature interaction module is embedded to facilitate bidirectional interaction and fusion between the local features output by the multi-scale convolution enhancement module and the global features output by the EVA-02 backbone network at the beginning and end of each stage.

[0019] Furthermore, the operation of the multi-scale convolution enhancement module includes:

[0020] Receive multi-scale feature maps with different spatial resolutions from the convolutional branches;

[0021] The multi-scale feature maps are flattened and stitched together into a feature sequence;

[0022] The feature sequence is linearly projected to reduce its dimensionality, and the dimensionality-reduced features are divided into M groups along the channel dimension;

[0023] The M groups are processed in parallel by using M depth-separable convolutional layers with different kernel sizes;

[0024] The M sets of features processed by parallel convolution are concatenated and restored to the original channel dimension through linear projection, outputting the enhanced multi-scale features.

[0025] Furthermore, the execution process of the collaborative feature interaction module includes:

[0026] Fusion steps: The single-scale global features from the EVA-02 backbone network and the multi-scale features from the multi-scale convolutional enhancement module are added element-wise to the feature map with compatible spatial resolution. The fused multi-scale feature map is used as the initial aggregation feature.

[0027] Unification Steps: The initial aggregated features are input into a multi-scale self-attention module for processing to unify features from different modalities and output a deep fusion multi-scale feature set;

[0028] Bidirectional injection steps:

[0029] In the forward injection, the multi-scale feature set after deep fusion is scale-aligned and then weighted and fused with the current stage features of the EVA-02 backbone network to update the backbone network features.

[0030] In the backward injection, the backbone network features, which are rich in global context and have been processed by the Transformer module in the current stage, are fused again with the features of the convolutional branch to update the convolutional branch features.

[0031] Furthermore, the core module of the EVA-02 backbone network is the Transform Vision module, which uses a SwigLU structure as a feedforward network and employs rotational position embedding.

[0032] Furthermore, prior to step S1, a step for preparing model training data is included, specifically:

[0033] The original poplar leaf images are processed through a two-stage lesion segmentation and infection rate quantification process to automatically calculate the leaf infection rate of each image.

[0034] Based on a five-level quantitative grading standard established according to the statistical distribution of infection rate and pathological principles, the severity level of each image is labeled, thereby constructing a dataset for model training.

[0035] Furthermore, the two-stage lesion segmentation and infection rate quantification process includes:

[0036] Phase 1: Perform linear combination and feature subtraction operations across color channels on the background-normalized leaf image to segment out the lesion area;

[0037] Phase 2: Parallel execution of lesion pixel statistics and precise segmentation of leaf regions based on edge detection and contour simplification, and finally calculation of infection rate based on the ratio of lesion pixel area to leaf pixel area.

[0038] In a second aspect, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of the first aspects.

[0039] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0040] Fourthly, the present invention provides a hybrid deep learning device for ordered severity grading of poplar black spot disease, comprising:

[0041] The image acquisition module is used to acquire images of the poplar leaves to be tested.

[0042] An image preprocessing module is used to perform standardized preprocessing on the image;

[0043] The model inference module is loaded with a pre-trained PoplarFormer grading model, which is used to receive pre-processed images and output the severity level of poplar black spot disease.

[0044] The information output module is used to receive the severity level and present it to the user;

[0045] The PoplarFormer hierarchical model loaded by the model inference module is a hybrid neural network built on the EVA-02 visual Transformer backbone network, which includes:

[0046] A multi-scale convolutional enhancement module is configured to extract multi-scale local fine-grained features from the input image in parallel.

[0047] A collaborative feature interaction module is configured to bidirectionally fuse the local fine-grained features output by the multi-scale convolutional enhancement module with the global context features extracted by the EVA-02 backbone network at multiple stages of the network.

[0048] Furthermore, the PoplarFormer hierarchical model is obtained through a training process supervised by the CORAL loss function.

[0049] Furthermore, the multi-scale convolution enhancement module includes:

[0050] The feature stitching unit is used to receive and stitch feature maps of different scales from the convolutional branches;

[0051] The dimensionality reduction and grouping unit is used to perform linear projection dimensionality reduction on the stitched features and group them along the channel dimension;

[0052] A multi-branch convolutional unit consists of multiple parallel, depth-separable convolutional layers with different kernel sizes, used to process different feature groups separately.

[0053] The feature merging unit is used to concatenate the features processed by the multi-branch convolutional unit and linearly project them onto the original dimension.

[0054] Furthermore, the collaborative feature interaction module includes:

[0055] The feature aggregation unit is used to add multi-scale convolutional features to backbone network features element by element;

[0056] The multi-scale attention unit is a self-attention module that employs a multi-scale deformable attention mechanism, used to perform deep fusion of the multi-scale features output by the feature aggregation unit;

[0057] The feature routing unit is used to bidirectionally inject the fused features output by the multi-scale attention unit into the corresponding feature streams of the EVA-02 backbone network and the convolutional branch.

[0058] Furthermore, the device also includes a model training data preparation module, the preparation module comprising:

[0059] The infection rate quantification unit is used to process the original poplar leaf images and automatically calculate the infection rate through a two-stage lesion segmentation and infection rate quantification process.

[0060] The severity labeling unit is used to automatically label the severity level of images according to a preset five-level quantitative grading standard based on infection rate, in order to generate a training dataset.

[0061] Furthermore, the device is specifically integrated into one of the following devices: a drone, a field inspection robot, a handheld smart agricultural terminal, or a cloud server.

[0062] (III) Beneficial Effects

[0063] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0064] Significantly improved hierarchical accuracy and logical consistency: The core PoplarFormer model of this invention deeply integrates the local details of CNN and the global context of ViT through parallel multi-scale convolution enhancement modules and collaborative feature interaction modules. It also explicitly learns the hierarchical order by combining the CORAL ordered loss function, which jointly ensures the high accuracy and high logical consistency of the prediction results from both the feature extraction and optimization objectives.

[0065] Objectivity and standardization of the assessment process: Through an innovative two-stage lesion segmentation and infection rate quantification process and a data-driven five-level quantitative grading standard, subjective assessment is transformed into objective and reproducible scientific analysis, providing a unified assessment standard for the industry.

[0066] The invention boasts advanced technology and strong practicality: Experiments on a self-built benchmark dataset demonstrate that the proposed method significantly outperforms existing mainstream models in key metrics. The provided complete technical framework exhibits high transferability, can be deployed on various hardware platforms such as drones and robots, achieving a closed loop from algorithm to application, and possesses extremely high practical value and widespread application potential. Attached Figure Description

[0067] Figure 1 This is a schematic diagram of the overall process of a hybrid deep learning method provided in an embodiment of the present invention.

[0068] Figure 2 This is a detailed schematic diagram of the two-stage lesion segmentation and infection rate quantification (DPLS-IRQ) process in an embodiment of the present invention.

[0069] Figure 3 This is a schematic diagram of the overall architecture of the PoplarFormer hierarchical model in an embodiment of the present invention.

[0070] Figure 4This is a schematic diagram of the structure of the multi-scale convolution enhancement module (MSCEM) in an embodiment of the present invention.

[0071] Figure 5 This is a schematic diagram of the collaborative feature interaction module (SFIM) in an embodiment of the present invention.

[0072] Figure 6 This is a distribution map of the nuclear density estimation (KDE) of infection rate used to construct a five-level grading standard in an embodiment of the present invention.

[0073] Figure 7 This is a comparison of Grad-CAM visualization heatmaps between the PoplarFormer model and the baseline model in this embodiment of the invention.

[0074] Figure 8 This is a comparison of the classification confusion matrix between the PoplarFormer model and the baseline model in this embodiment of the invention.

[0075] Figure 9 This is a structural block diagram of a hybrid deep learning device provided in an embodiment of the present invention. Detailed Implementation

[0076] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0077] Example 1: A hybrid deep learning method for ordered severity grading of poplar black spot disease

[0078] This invention provides a hybrid deep learning method for ordered severity grading of poplar black spot disease. (Refer to...) Figure 1 The overall flowchart of this method illustrates the core of the technical solution of this invention, which can include two stages: offline training and online application. The offline stage mainly includes step S101 (preparing model training data) and step S102 (constructing and training the PoplarFormer grading model); the online application stage mainly includes step S103 (performing disease grading).

[0079] Step S101: Prepare model training data

[0080] The purpose of this step is to address the lack of a scientific and unified quantitative grading standard mentioned in the background art. This embodiment automatically calculates the infection rate through a two-stage lesion segmentation and infection rate quantification (DPLS-IRQ) process, and establishes a five-level quantitative grading standard based on this to construct the Poplar-BSR5 dataset.

[0081] In one specific implementation, the original images were collected in September 2023 at Malanghu Forest Farm, Sihong County, Jiangsu Province, China, totaling 4,955 high-resolution leaf images of 5760×4624 pixels.

[0082] To facilitate subsequent processing, the image can be normalized. For example, the U2-Net model can be used to accurately segment the leaves from the original black background and then re-synthesize them onto a pure white background.

[0083] Reference Figure 2 Execute the DPLS-IRQ procedure

[0084] Phase One (High-Precision Lesion Segmentation): The core is to utilize mathematical operations across color channels (such as...) (This involves) and feature subtraction. By subtracting an "accompanying image" (containing only the dilated background) from a "memory image" (containing both lesions and background), an image containing only lesion features is ultimately extracted.

[0085] Phase Two (Leaf Segmentation and Infection Rate Quantification): This phase executes two sub-tasks in parallel: (a) processing the lesion images output from Phase One. Pixel statistics were performed to obtain the lesion area. (b) Perform a separate contour reconstruction algorithm on the original image to obtain the leaf mask with high accuracy. Specifically, the contour reconstruction algorithm includes: converting the RGB image to the HSV color space, and using a normalized white background. The characteristics of saturation and brightness The threshold is used to separate the leaves; then, to capture the fine geometry, the Canny edge detection algorithm is used. In a preferred implementation, the Gaussian blur kernel size is [value missing]. The Canny low threshold is 50, and the high threshold is 150. Finally, to smooth noise and preserve edge structure, the Douglas-Peucker line simplification algorithm is used to refine the contour, with a tolerance of [missing information]. Preferably, it is a small proportion of the total perimeter, for example The leaf pixel area is obtained through pixel statistics. Finally, according to the formula Calculate the accurate infection rate.

[0086] Establish a five-level quantitative grading standard. (Refer to...) Figure 6Kernel density estimation (KDE) analysis was performed on 4955 infection rate values ​​generated by the DPLS-IRQ workflow. Based on statistical distribution (e.g., the 20th, 40th, 60th, and 80th percentiles, which are 4.84%, 9.08%, 14.71%, and 23.80%, respectively) and combined with pathological corrections, a reasonable positive number was established, such as σ=0.5 or σ=1. Alternatively, for greater stability, σ was not specified, indicating that it can be automatically calculated based on kernel size. A five-level quantitative grading standard, as shown in Table 1, was established based on pathological principles (e.g., 5% for initial infection, 25% for severe lesions).

[0087] Table 1: Quantitative Grading Standards for Poplar Black Spot Disease (Level 5)

[0088]

[0089] Based on this standard, 4955 images were labeled to construct the Poplar-BSR5 dataset. The detailed category distribution of this dataset is shown in Table 2.

[0090] Table 2: Category Distribution and Statistics of the Poplar-BSR5 Dataset

[0091]

[0092] Step S102: Construct and train the PoplarFormer hierarchical model

[0093] This step aims to build and optimize the PoplarFormer model.

[0094] (1) Model building

[0095] Reference Figure 3 PoplarFormer is a two-branch hybrid network that includes three core components:

[0096] (a) EVA-02 backbone network (main branch): Employing an advanced EVA-02-Small variant, it is responsible for capturing global contextual information. Its core TrV module uses SwiGLU FFN and Rotational Position Embedding (RoPE), providing powerful visual representation capabilities.

[0097] (b) Multi-scale Convolutional Enhancement Module (MSCEM): As a parallel convolutional branch, it is used to capture local fine-grained lesion features that are easily overlooked in the ViT backbone. (See reference...) Figure 4 Its operation includes: receiving multi-scale feature maps. Flatten and splice them into a feature sequence After dimensionality reduction via linear projection, it is divided along the channel dimension into Group; by Different core sizes (e.g.) The depthwise separable convolutions (DWConv) are processed in parallel; the results are concatenated and up-dimensionalized via linear projection to output enhanced multi-scale features. .

[0098] (c) Collaborative Feature Interaction Module (SFIM): used for... Each stage (in this embodiment) In this process, the features of the two branches mentioned above are fused bidirectionally. (Refer to...) Figure 5 Its execution process includes:

[0099] Fusion steps: Integrating backbone features (1 / 16 resolution) Features compatible with resolution in MSCEM output (1 / 16 resolution) Element-wise addition is performed to obtain the initial aggregated features. .

[0100] Unified steps: To address modal bias, aggregate features The input is processed by a Transformer module employing Multi-Scale Deformable Attention (MSSA), and the output is a deeply fused feature set. .

[0101] Bidirectional injection steps: in the stage The beginning (forward injection) will (After scale alignment, it is denoted as) Through weighted fusion ( Injected into the backbone network to update global features ; in the stage At the end (backward injection), the updated backbone features are fused with the convolutional branch features again to produce the output. Then it was injected into the CNN branch ( ), update convolutional branch features .

[0102] (2) Model Training

[0103] Training is performed on the Poplar-BSR5 dataset constructed in step S101 (divided into training, validation, and test sets in a 7:2:1 ratio).

[0104] Preprocessing and enhancement: The image is scaled and center-cropped. Resolution, and using the mean of the EVA model family. and standard deviation Normalization was performed. The training set employed methods including random horizontal flipping. Vertical flip Online data augmentation strategies, including rotation and color jitter.

[0105] Optimizer and Loss Function: To address the issue of ignoring orderliness mentioned in the background section, this embodiment employs the CORAL loss function for supervised optimization. CORAL will... The rank problem is reconstructed as For a binary classification task with shared weights, the "rank monotonicity" guarantees that a penalty is imposed on a "slight" misclassification as "severe" than on a misclassification as "moderate," thereby enabling the model to learn the inherent logical order of the levels.

[0106] Training parameters: using the AdamW optimizer Weight decay of 0.05, training for 50 epochs. Cosine annealing learning rate and 10% warm-up were used. Differential learning rate was used: the pre-trained EVA-02 backbone used... The low learning rate and random initialization of the MSCEM and SFIM modules are used It has a relatively high learning rate. The effective batch size is 64.

[0107] Step S103: Perform disease severity classification

[0108] In the application phase, images of the poplar leaves to be tested are acquired (e.g., via drone, robot, or handheld device), and the same preprocessing described in step S102(2) is performed. The preprocessed image is then input into the PoplarFormer grading model trained in step S102(2). The model performs forward propagation and outputs a prediction result corresponding to one of the five severity levels (1_Very_Mild to 5_Severe) in Table 1.

[0109] Verification of the beneficial effects of Example 1

[0110] To demonstrate the effectiveness of the method described in this embodiment, it was evaluated on the Poplar-BSR5 test set. Evaluation metrics included accuracy, weighted F1 score, number of model parameters (M), and mean absolute error (MAE), which measures the logical consistency of ordered tasks.

[0111] (1) Comparative analysis with existing technologies

[0112] The PoplarFormer of this invention is compared with mainstream models such as Swin-T, ConvNeXt-V2, CoAtNet-0, and EVA-02 baselines. The results are shown in Table 3.

[0113] Table 3: Performance comparison of different models on the Poplar-BSR5 test set

[0114]

[0115] As shown in Table 3, the PoplarFormer of this invention achieves optimal performance across all metrics. Its accuracy reaches 83.80%, significantly higher than the baseline EVA-02 (81.40%). Particularly noteworthy is the critical MAE metric, where PoplarFormer's error is only 0.1633, a 14.1% reduction compared to the baseline's 0.1901, demonstrating the significant advantage of this invention in maintaining logical consistency.

[0116] (2) Ablation study (verifying the contribution of each component)

[0117] To systematically verify the independent contributions and synergistic effects of the various innovative components in this invention, this embodiment conducted a series of ablation studies.

[0118] First, to verify the effectiveness of the MSCEM and SFIM modules in the PoplarFormer hybrid architecture of this invention, a progressive architecture ablation experiment was conducted. See Table 4.

[0119] Table 4: Ablation Study of Architectural Components

[0120]

[0121] As shown in Table 4, the baseline model (EVA-02) achieved an accuracy of 81.40%. Adding only the MSCEM module and using simple summation (Sum) fusion resulted in limited performance improvement (81.60%). However, after introducing the SFIM module of this invention, even with unidirectional fusion (C... (V), accuracy also improved to 82.00%. When using full bidirectional fusion (C) When V), the model performance reaches its optimal level (82.40%). This strongly demonstrates the superiority of the proposed MSCEM module and SFIM bidirectional fusion architecture.

[0122] Secondly, to verify the synergistic effect of the hybrid architecture and the CORAL loss function of this invention, combined ablation experiments were conducted. See Table 5.

[0123] Table 5: Ablation Study of Independent and Combined Contributions of Architectural Modules to CORAL Loss

[0124]

[0125] As shown in Table 5, introducing CORAL loss alone (row 2) or introducing the architecture module alone (row 3) can improve performance, but the performance is optimal when both are combined (row 4, i.e., the complete PoplarFormer). This demonstrates the powerful synergy between architectural innovation and the ordered loss optimization objective in this invention.

[0126] Finally, hyperparameter analysis was conducted to determine the preferred configuration parameters for the innovative module of this invention.

[0127] Referring to Table 6, the kernel size for the Multi-Scale Convolutional Enhancement Module (MSCEM) is specified. Tests were conducted.

[0128] Table 6: MSCEM Core Size Analysis

[0129]

[0130] Referring to Table 7, the number of interactions with the Collaborative Feature Interaction Module (SFIM) is as follows. Tests were conducted.

[0131] Table 7: Number of SFIM Interactions Analysis

[0132]

[0133] As shown in Tables 6 and 7, when the kernel size combination is {3, 5} and the number of interactions... At this point, the model achieves an optimal balance between accuracy and parameter count. Therefore, this embodiment uses {3, 5} and As the preferred configuration for MSCEM and SFIM.

[0134] (3) Visualization Analysis

[0135] Reference Figure 7 The attention of PoplarFormer and the baseline model was compared using Grad-CAM heatmaps. The baseline model's attention was diffuse and incomplete, often missing lesions. PoplarFormer, by incorporating both local and global features, allows its attention activation region to be precisely and clearly located on fine-grained pathological features.

[0136] Reference Figure 8 Compare the confusion matrix. Baseline model ( Figure 8 a) Severe confusion exists between adjacent categories (e.g., 3_Mild_Moderate and 4_Moderate) (20 cases). The PoplarFormer of this invention ( Figure 8 b) The main diagonal is denser, and off-diagonal errors are significantly reduced (only 8 cases between 3_Mild_Moderate and 4_Moderate), demonstrating its stronger discrimination ability and logical consistency.

[0137] In summary, the method in this embodiment solves the problem of quantitative evaluation through an innovative DPLS-IRQ process and a five-level standard; it effectively integrates local and global features through the MSCEM and SFIM modules; and it ensures the logicality of the prediction through CORAL loss. Its beneficial effects (as shown in Tables 3 to 4) are further illustrated in these examples. Figure 7 It is significantly superior to existing technologies.

[0138] Example 2: A hybrid deep learning device for ordered severity grading of poplar black spot disease

[0139] This invention also provides a hybrid deep learning apparatus for implementing the above-described method. (Refer to...) Figure 9 This apparatus is a system block diagram for performing the method described in Embodiment 1. The apparatus may be an electronic device such as a server, personal computer, mobile terminal, drone, or embedded system. The apparatus includes:

[0140] Image acquisition module: Used to acquire images of the poplar leaves to be tested. In specific implementations, this can be a CMOS / CCD camera connected to the device, a drone-mounted aerial camera, or an interface for loading and storing images.

[0141] Image preprocessing module: used to perform normalization preprocessing on the image, such as the size adjustment (e.g., cropping) described in step S103 of Embodiment 1. ) and normalization (using the mean and standard deviation of the EVA model family).

[0142] Model inference module: This is the core of the device, used to receive preprocessed images and output the severity level of poplar black spot disease. This module is loaded with the PoplarFormer grading model trained according to Example 1. The model inference module specifically includes:

[0143] (a) EVA-02 backbone network: used to extract global contextual features of images.

[0144] (b) Multi-Scale Convolutional Enhancement Module (MSCEM): Configured to extract multi-scale local fine-grained features from the input image in parallel. Its internal structure can be found in [reference needed]. Figure 4 As described in Example 1 (including feature splicing unit, dimensionality reduction and grouping unit, multi-branch convolution unit and feature merging unit), and preferably using the {3, 5} kernel size combination as verified in Example 1 and Table 6.

[0145] (c) Collaborative Feature Interaction Module (SFIM): Configured to bidirectionally fuse the local fine-grained features output by MSCEM with the global contextual features extracted by the EVA-02 backbone network at multiple stages of the network. Its internal structure can be found in [reference needed]. Figure 5As described in Example 1 (including a feature aggregation unit, a multi-scale attention unit, and a feature routing unit, and preferably employing the features verified in Example 1 and Table 7), Secondary interaction configuration.

[0146] Information output module: Used to receive the severity level (e.g., an internal code "4") from the model inference module and convert it into user-readable information (e.g., the text "4_Moderate" or "Moderate"), which is then presented via a display screen, printer, or user interface.

[0147] In an optional embodiment, the device may further include a model training data preparation module (such as...). Figure 9 (As shown in the dashed box), this module is used to perform the dataset construction process described in step S101 of Example 1. This module includes:

[0148] (a) Infection rate quantification unit: used for execution Figure 2 The DPLS-IRQ process shown automatically calculates the infection rate.

[0149] (b) Severity labeling unit: used to automatically label the severity level of images according to the five-level quantitative grading standard shown in Table 1, in order to generate a dataset for model training or updating.

[0150] The device can be specifically integrated into: drones and field inspection robots for large-scale automated inspections; handheld smart agricultural terminals for rapid on-site diagnosis; or deployed on cloud servers to provide online analysis services.

[0151] The present invention also provides an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor executes the method described in Example 1 when it runs.

[0152] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in Embodiment 1.

[0153] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A hybrid deep learning method for ordered severity grading of poplar black spot disease, characterized in that, Includes the following steps: Step S1: Acquire an image of the poplar leaf to be tested, and preprocess the image; Step S2: Input the preprocessed image into the trained PoplarFormer grading model, and the model outputs the severity level of poplar black spot disease; The PoplarFormer hierarchical model is a hybrid neural network built on the visual Transformer backbone network. It extracts local fine-grained features of the image through a parallel multi-scale convolution enhancement module and fuseds the local fine-grained features with the global context features extracted by the backbone network through a collaborative feature interaction module. Furthermore, the PoplarFormer grading model is optimized during the training phase using an ordered regression loss function to learn the ordered relationship between severity levels.

2. The method according to claim 1, characterized in that, The PoplarFormer hierarchical model is constructed in the following way: Construct a dual-branch network, which includes the EVA-02 backbone network as the main branch and a convolutional branch as the parallel branch; The multi-scale convolution enhancement module is set in the convolution branch to generate a multi-scale feature pyramid from the input image and extract local spatial features of multiple receptive fields in parallel using depth-separable convolutional layers with different kernel sizes. In multiple consecutive Transformer module stages of the EVA-02 backbone network, the collaborative feature interaction module is embedded to facilitate bidirectional interaction and fusion between the local features output by the multi-scale convolution enhancement module and the global features output by the EVA-02 backbone network at the beginning and end of each stage.

3. The method according to claim 2, characterized in that, The execution process of the collaborative feature interaction module includes: Fusion step: The global features from the backbone network and the multi-scale features from the multi-scale convolutional enhancement module are fused element-wise by addition; Unified steps: Input the fused features into a multi-scale self-attention module for processing, and output a deep fused multi-scale feature set; Bidirectional injection steps: In the forward injection, the multi-scale feature set after deep fusion is scale-aligned and then weighted and fused with the current stage features of the backbone network to update the backbone network features. In the backward injection, the updated backbone network features are fused with the features of the convolutional branches again to update the convolutional branch features.

4. The method according to claim 1, characterized in that, The ordered regression loss function is the CORAL loss function.

5. The method according to claim 1, characterized in that, Before step S1, there is also a step for preparing model training data, specifically: The original poplar leaf images are processed through a two-stage lesion segmentation and infection rate quantification process to automatically calculate the leaf infection rate of each image. Based on a five-level quantitative grading standard established according to the statistical distribution of infection rate and pathological principles, the severity level of each image is labeled, thereby constructing a dataset for model training.

6. A hybrid deep learning device for ordered severity grading of poplar black spot disease, characterized in that, include: The image acquisition module is used to acquire images of the poplar leaves to be tested. An image preprocessing module is used to perform standardized preprocessing on the image; The model inference module is loaded with a pre-trained PoplarFormer grading model, which is used to receive pre-processed images and output the severity level of poplar black spot disease. The information output module is used to receive the severity level and present it to the user; The PoplarFormer hierarchical model loaded by the model inference module is a hybrid neural network built on the EVA-02 visual Transformer backbone network, which includes: A multi-scale convolutional enhancement module is configured to extract multi-scale local fine-grained features from the input image in parallel. A collaborative feature interaction module is configured to bidirectionally fuse the local fine-grained features output by the multi-scale convolutional enhancement module with the global contextual features extracted by the EVA-02 backbone network at multiple stages of the network. Furthermore, the PoplarFormer hierarchical model is obtained through a training process supervised by the CORAL loss function.

7. The apparatus according to claim 6, characterized in that, The multi-scale convolution enhancement module includes: The feature stitching unit is used to receive and stitch feature maps of different scales from the convolutional branches; The dimensionality reduction and grouping unit is used to perform linear projection dimensionality reduction on the stitched features and group them along the channel dimension; A multi-branch convolutional unit consists of multiple parallel, depth-separable convolutional layers with different kernel sizes, used to process different feature groups separately. The feature merging unit is used to concatenate the features processed by the multi-branch convolutional unit and linearly project them onto the original dimension.

8. The apparatus according to claim 6, characterized in that, The collaborative feature interaction module includes: The feature aggregation unit is used to add multi-scale convolutional features to backbone network features element by element; The multi-scale attention unit is a self-attention module that employs a multi-scale deformable attention mechanism, used to perform deep fusion of the multi-scale features output by the feature aggregation unit; The feature routing unit is used to bidirectionally inject the fused features output by the multi-scale attention unit into the corresponding feature streams of the EVA-02 backbone network and the convolutional branch.

9. The apparatus according to claim 6, characterized in that, The device further includes a model training data preparation module, the preparation module comprising: The infection rate quantification unit is used to process the original poplar leaf images and automatically calculate the infection rate through a two-stage lesion segmentation and infection rate quantification process. The severity labeling unit is used to automatically label the severity level of images according to a preset five-level quantitative grading standard based on infection rate, in order to generate a training dataset.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 5.

Images