Intelligent plant health inspection method and system based on mobile robot

Abstract

The application discloses a kind of plant health intelligent inspection method and system based on mobile robot, it is related to plant health detection and mobile robot inspection technical field.The method includes: using the high-resolution visible light holder of four-legged robot, obtains garden vegetation image;Based on Swin-Transformer architecture, feature extraction is carried out to visible light image;Through joint feature representation learning network, the class information of plant pest and the hazard severity level of plant pest are mapped to unified fine-grained classification space, and the synchronous identification and situation quantitative evaluation of vegetation disease state are realized.CNN texture extraction and transformer global modeling are deeply coupled in the application, the feature extraction ability of hidden lesion under complex background is improved, and the end-to-end integrated perception of disease class and degree classification is realized.The application relies on four-legged robot, has the characteristics such as strong terrain adaptability, multi-task output cooperation, and is suitable for intelligent garden monitoring.

Description

Technical Field

[0001] This invention relates to the field of plant health detection and mobile robot inspection technology, and in particular to a plant health intelligent inspection method and system based on mobile robots, which is applicable to fine-grained classification of plant diseases and quantitative assessment of health status in community green belts, parks and urban ecological spaces. Background Technology

[0002] In landscaping, agriculture, and other scenarios, the health status of plants directly affects the landscape effect, crop yield, and ecological benefits. Pests and diseases, as well as nutrient deficiencies, are the main causes of poor plant growth. Timely and accurate health inspections are a core aspect of plant maintenance. Currently, traditional plant health inspections rely heavily on manual on-site observation, which suffers from low efficiency, strong subjectivity, and high missed detection rates. Furthermore, limitations in the scope and expertise of manual inspections make it difficult to achieve large-scale, routine, and accurate detection. While some existing technologies employ deep learning-based detection schemes based on visible light images, they still face the following technical bottlenecks: First, insufficient feature extraction capabilities. When processing complex garden backgrounds (such as dappled light and shadow, and partial occlusion), single convolutional neural networks (CNNs) often struggle to capture the spatial semantic features of fine-grained lesions due to limited receptive fields. Second, a lack of semantic modeling. Traditional models struggle to deeply correlate the overall health status of plants with local pathological evolution, resulting in insufficient accuracy and generalization ability in the quantitative assessment of disease severity, failing to meet the needs of smart gardens for precise health inspections.

[0003] Chinese patent document CN115187943A discloses "An Intelligent Path Planning Method for Inspection Robots Integrating GPR and DWA Algorithms." This method divides a grid map based on inspection nodes and employs block-based path planning. The algorithm has a short runtime and can quickly plan the globally optimal path. It integrates simulated plant growth algorithms and a dynamic window method to achieve intelligent local path planning in complex dynamic environments, effectively handling situations with obstacles. However, this technical solution has shortcomings in intelligent monitoring of vegetation health: First, its path planning and disease diagnosis functions have low coupling, failing to deeply integrate vegetation pathological semantic features with the inspection workflow. Second, the technology uses a single visual feature extraction method, making it difficult to handle complex and variable lighting and shading scenarios in community gardens, and unable to effectively model the spatial relationship between vegetation micro-texture features and macro-canopy structure, resulting in poor performance in fine-grained disease classification and quantitative assessment of severity. Furthermore, the existing technical solution lacks refined modeling of the evolutionary laws of vegetation pathological phenotypic features, making it difficult to achieve integrated and intelligent inspection of disease identification and severity quantification. Therefore, there is an urgent need in this field for a more intelligent and accurate robotic inspection technology solution that combines visual semantic modeling with multi-task collaborative perception to solve the problem of insufficient accuracy and depth of vegetation monitoring in complex garden environments. Summary of the Invention

[0004] Purpose of the invention: To provide a method and system for intelligent inspection of plant health based on a mobile robot, which uses Swin-Transformer visual feature collaborative perception to identify abnormal areas in images and simultaneously output the type of abnormality and the degree of health damage.

[0005] Technical Solution: To achieve the above objectives, one of the objectives of this invention is to disclose a plant health intelligent inspection method based on a mobile robot, comprising the following steps: S1 Data Acquisition: A mobile robot equipped with an RGB imaging unit, an inertial measurement unit (IMU), and a global navigation satellite system (GNSS) module is deployed in a green area. The initial inspection path is set and parameters are configured based on the geographic information system (GIS) map. The robot moves autonomously through simultaneous localization and mapping (SLAM) technology, simultaneously acquiring visible light images, multispectral images, environmental metadata, and pose information. All data is stored in a standardized manner after adding a unified timestamp. S2 Data Feature Construction: The visible light image acquired in step S1 is reconstructed. The image is divided into several pixel blocks by the PatchPartition module and mapped into a multi-dimensional visual feature vector by the Patch Embedding layer. The texture edge features and global spatial distribution features of the image are extracted by the preprocessing operator and multi-scale aggregation is performed along the channel dimension to generate a feature dataset with hierarchical visual semantics. S3 Annotated Dataset Construction: Following steps S1 and S2, sampling was performed under different seasons, weather conditions, and plant growth stages to obtain a multi-time period visual feature dataset. Professionals then used a dual-annotation + cross-verification process to annotate the samples in the multi-time period visual dataset with pest and disease categories, nutrient deficiency types, and health severity levels, thus constructing a structured fine-grained labeled dataset. S4 Training of the Swin-Transformer-based dual-stream sensing model: The end-to-end multi-task plant health analysis model is trained using the structured labeled dataset described in step S3. Through the multi-scale self-attention mechanism of the Swin Transformer, the identification of abnormal vegetation areas, the classification of abnormal causes, and the grading of health status are realized. After the perception model is trained, a mobile robot performs inspections, inputting RGB images into the SwingTransformer encoder and outputting plant disease and pest categories, nutrient deficiency types, and health damage levels.

[0006] Further, in step S4, the RGB image is input into the Swin Transformer encoder. First, the image is divided into 4×4 pixel blocks by the Patch Partition module, and then linearly mapped into a feature sequence by the Patch Embedding layer. ,in The feature sequence passes through four cascaded stages (Stages 1-4), each stage containing several Swin Transformer Blocks and an optional Patch Merging layer (except for Stage 4), where: The Swin Transformer Block extracts local-global features based on window multi-head self-attention (W-MSA) and sliding window multi-head self-attention (SW-MSA); The Patch Merging layer downsamples the feature map resolution to half of its original value while doubling the number of channels, thereby constructing a multi-scale, hierarchical feature pyramid. The downsampling factors were 4×, 8×, 16×, and 32×, respectively, to achieve deep semantic modeling of the subtle texture of lesions to the overall spread trend of vegetation.

[0007] Furthermore, the training process of the Swin-Transformer-based dual-stream sensing model described in step S4 includes: S4.1 Hierarchical visual semantic mapping utilizes the Patch Partition layer to map the preprocessed RGB image. Cut into , and The input image consists of a sequence of pixel blocks of height and width, respectively; each pixel block is mapped to a feature vector through a Patch Embedding layer. The feature sequence X enters a Swin Transformer encoder consisting of four cascaded stages (Stage 1-4); each stage contains several Swin Transformer Block modules, which achieve step-by-step downsampling of feature resolution through a Patch Merging layer. By progressively increasing the feature channel dimension, a multi-scale, hierarchical feature pyramid set is constructed. This pyramid structure enables deep semantic modeling from the subtle textures of local lesions to the spatial spread trend of the entire plant vegetation, providing an input basis for subsequent fine-grained disease perception. S4.2 Window Attention Feature Mining: Within each Stage, pathological features are mined using a moving window self-attention mechanism (SW-MSA). For input features... The calculation process is as follows: S4.2.1 Linear Feature Projection: Projecting the segmented Patch sequence Perform linear projection: ;;in, The weight matrix is ​​a learnable matrix. For the dimension of attention head, The total number of pixels (patches) within the window; S4.2.2 Relative Position Offset and Attention Calculation: Introducing a two-dimensional relative position offset matrix. Calculate the global long-range semantic correlation degree: ;in The enhanced feature sequence, aggregated through a self-attention mechanism, represents the strength of the semantic association between patches. These are the query matrix, key matrix, and value matrix, each with a dimension of [missing information]. ; Key matrix The transpose of the matrix is ​​used to perform spatial dot product operations to capture the correlation between image patches; This is the scale scaling factor. This factor represents the dimension of the attention head; scaling using this factor can effectively prevent the value after the dot product operation from becoming too large. The function enters the gradient flat region, ensuring training stability; This is a two-dimensional relative position bias matrix; this matrix introduces the two-dimensional geometric topological relationship of plant lesions on the image surface into the feature aggregation process by encoding the relative spatial distance between patches within the window. The normalized exponential function, through calculation, maps the correlation between features to... The probability distribution weights of the intervals represent the degree of attention each Patch node pays to the current center point; This represents the total number of patches contained within the window, i.e., the length of the feature sequence. S4.3 Multi-branch Parallel Decoding and Collaborative Decision-Making: To determine the overall health status of the plant, a Global Average Pooling (GAP) operation is first performed to aggregate spatial features into a single semantic vector. : ;in, The aggregated global feature vector has a dimension of . This characterizes the overall pathological semantic state of vegetation images; For height; Width; For feature maps in spatial coordinates The eigenvector at that location; It is a spatial normalization factor. By averaging all pixels in the feature map, it achieves feature smoothing and dimensionality reduction in the spatial dimension, eliminating the influence of the specific location offset of the lesion in the image on the discrimination result. S4.3.1 Fine-grained classification branch, outputting the probability distribution of disease categories. , ;in, This represents the preset total number of disease types; The learnable weight matrix for the classification branch has dimensions of . This is used to map the global semantic feature space to the disease category decision space; Let be the bias vector for the classification branch, with dimension . This is used to adjust the output threshold for disease categories; The global semantic feature vector output by the Global Average Pooling (GAP) layer has a dimension of . This characterizes the overall health semantics of vegetation; S4.3.2 Hazard level classification branch, outputs the probability distribution of severity levels. ;in, The total number of severity levels is determined by a probability distribution. The operation realizes the quantitative judgment of vegetation health status; W1 and W2 are the weight matrices of the first and second fully connected network layers, respectively, used to learn the nonlinear mapping from the feature space to the health level space; b1 and b2 are the bias terms of the two fully connected network layers, respectively, used to adjust the activation threshold of perception decision; ReLU(⋅) is a linear rectified activation function, used to introduce nonlinear feature expression ability and enhance the model's depth of mining pathological evolution features; Softmax(⋅) is a normalized exponential function, which maps the output layer to a probability distribution vector of K levels; The S4.4 end-to-end joint optimization strategy defines the total loss function as the weighted sum of the loss from disease type identification and the loss from severity classification, and uses the backpropagation algorithm to update network parameters synchronously. Through multi-task collaborative training, the model achieves deep coupling and dynamic optimization of disease characterization and severity measurement.

[0008] Furthermore, the dual-stream sensing branch described in step S4 is implemented by constructing a fine-grained classification branch and a health level assessment branch in parallel at the decoding end; Fine-grained classification branch: A global average pooling layer (GAP) is used for global spatial semantic aggregation. After mapping through a fully connected layer, a fine-grained joint probability distribution of disease type and severity is output through a softmax activation function. ;in, This represents the total number of disease types. This represents the severity level; this branch achieves accurate qualitative identification of the causes of diseases through a composite mapping of category attributes and evolution degree. Health level assessment branch: Employs a lightweight multilayer perceptron (MLP) architecture, independently receiving global semantic features. Through two fully connected layers and the ReLU activation function, a multi-dimensional quantitative feature vector of vegetation health status is output. Finally, a softmax layer outputs a discrete rank distribution of three dimensions: disease, pests, and nutrient stress. This branch achieves quantitative analysis of the overall health status of plants through feature decoupling.

[0009] Furthermore, the end-to-end joint optimization strategy described in step S4.4 includes: S4.4.1 Constructing the composite total loss function Joint and collaborative optimization of multi-task-aware branches; , in, The weighting coefficients are determined through a grid search on the validation set to balance them. S4.4.2 Classification of Disease Types and Losses The label-smooth cross-entropy loss is used, as shown in the following formula: , in, For the total sample size, This represents the total number of preset disease categories; For the smoothed true label, These are the probability values ​​for the types of diseases predicted by the model. 4.4.3 Severity Classification of Losses Multi-class cross-entropy loss is used to optimize the quantitative perception of vegetation health status. , in, The total number of preset severity levels, For the total sample size, For the corresponding severity level, use one-hot encoded real labels. The predicted rank probability value by the model; The collaborative optimization mechanism synchronously updates the weight parameters of the encoder and the decoding sensor head through the backpropagation algorithm, enabling the model to simultaneously possess the ability to distinguish disease types and quantitatively assess health levels in a single RGB mode, effectively enhancing the comprehensive representation accuracy of the feature backbone for vegetation pathological evolution characteristics.

[0010] Furthermore, the environmental metadata in step S1 includes temperature, humidity, and light intensity; the effective observation distance of the multispectral image unit is 0.5–3 meters; and the configuration parameters include inspection speed, detection confidence threshold, and task priority strategy based on vegetation density.

[0011] Furthermore, in step S3: the pest and disease categories include fungal diseases, insect pests, and bacterial diseases; the nutrient deficiency types include nitrogen deficiency, iron deficiency, and magnesium deficiency; and the health severity levels include normal, mild disease-related malnutrition, and severe disease-related malnutrition.

[0012] The second objective of this invention is to disclose a plant health intelligent inspection system based on a mobile robot, used to implement the above-mentioned method, including a hardware integration module and a software algorithm module; The hardware integration module is a mobile robot platform equipped with multiple sensors, including the mobile robot body, a visible light imaging unit, an IMU module, a GNSS module, and an autonomous navigation module integrated into the body. The visible light imaging unit is used to acquire RGB images of vegetation in real time during the inspection process, and the autonomous navigation and SLAM module is used to plan the inspection path and locate in real time. The software algorithm module includes a data acquisition module, an encoding module, and a visual-semantic collaborative perception module. The data acquisition module controls the hardware integration module to acquire visible light images in real time and simultaneously add pose and timestamp information. The encoding module uses Swing Transformer to extract vegetation texture and semantic features. The visual-semantic collaborative perception module constructs a multi-task joint perception network based on hybrid features to achieve integrated identification and quantitative assessment of vegetation disease types and health severity, and outputs diagnostic results through a visualization terminal.

[0013] Furthermore, the encoding module incorporates a multi-task decoding head and a loss calculation unit; the multi-task decoding head includes a detection head, a segmentation head, and a regression head, which respectively realize abnormal area location and classification, disease area precise location, and health status assessment; the loss calculation unit incorporates detection loss, nutrient status regression loss, and total loss functions, supports determining loss weights through validation set grid search, and dynamically fine-tunes them according to disease type.

[0014] The beneficial effects of this invention are: 1. The intelligent plant health inspection method based on a mobile robot provided by this invention utilizes multi-scale disease feature encoding and patch mapping. It acquires visible light images using a high-resolution camera mounted on a quadruped robot, divides the input image into multiple micro-patterns using PatchPartition, and constructs a multi-dimensional feature pyramid containing local lesion textures and global plant morphology using a Patch Embedding layer. This module addresses scale fluctuations and noise interference in garden environments by achieving structured representation of multi-scale features at the encoder input. This ensures that the model can fully preserve the structural integrity of fine-grained plant disease information and macroscopic morphological features when dealing with complex lighting backgrounds, providing a foundation for subsequent in-depth analysis.

[0015] 2. The intelligent plant health inspection method based on mobile robots provided in this invention introduces a Swing-Transformer structure to achieve complementary extraction and collaborative modeling of multi-dimensional features of diseases. Utilizing the local translation invariance and efficient texture extraction capabilities of the convolutional module, it accurately captures subtle deformations and discoloration features at the edges of lesions; simultaneously, it employs a global self-attention mechanism to capture the long-range spatial dependencies and correlation trends between lesions in different locations. This hybrid modeling approach effectively overcomes the lack of global pathological feature capture capability of a single convolutional model in complex occlusion environments, achieving adaptive dynamic description of disease phenotypic features and significantly improving the robustness of disease feature representation.

[0016] 3. The intelligent plant health inspection method based on mobile robots provided by this invention unifies disease classification and severity grading modeling into a joint feature prediction task based on a shared hybrid feature backbone. Through end-to-end training and multi-task collaborative optimization mechanisms, the characteristics of plant disease types and severity grading features achieve mutual constraints and co-evolution within the shared backbone, effectively avoiding semantic loss caused by task fragmentation in traditional phased monitoring pipelines, and realizing integrated perception of disease discrimination and severity quantification. Relying on the high-dimensional semantic expression of the hybrid feature backbone, it accurately identifies the types of pests and diseases or nutrient stress types causing local abnormalities in vegetation; based on the feature mapping of lesion coverage and pathological evolution patterns, it outputs the damage level of health status using ordinal classification.

[0017] 4. The Swin-Transformer architecture constructed in this invention effectively solves the feature extraction bottleneck of a single convolutional neural network in complex natural environments. Through in-depth mining of lesion texture details by the convolutional module and long-range modeling of disease morphological evolution by the Transformer global attention mechanism, the model has stronger noise resistance and feature discrimination when dealing with complex light and shadow interference, partial leaf occlusion and background redundancy information commonly found in community gardens, thus achieving accurate localization and identification of early-stage hidden diseases.

[0018] 5. This invention constructs a multi-task collaborative perception and joint optimization architecture, integrating the identification of vegetation disease causes and severity classification into a unified visual semantic perception network. Based on a shared hybrid feature backbone, this architecture executes fine-grained disease classification and hazard level rating tasks in parallel. Through joint feature space mapping and end-to-end collaborative training, it achieves integrated decision output for vegetation health status, effectively solving the defects of fragmented pathological semantic features and information loss in traditional phased monitoring pipelines. At the training strategy level, the model constructs a composite total loss function consisting of disease type classification loss and health level assessment loss. Multi-task joint optimization drives the encoder to synchronously learn category features and evolutionary patterns. Simultaneously, by setting dynamic loss weight coefficients for different disease categories, the gradient imbalance problem in the multi-task learning process is effectively alleviated. Furthermore, the introduction of label smoothing strategies and multi-class cross-entropy optimization significantly improves the model's generalization ability, recognition accuracy, and perception robustness in complex garden backgrounds, providing high-precision quantitative decision support for disease tracing and intelligent management in precision agriculture.

[0019] 6. The intelligent plant health inspection system based on mobile robots provided by this invention deeply integrates mobile robot autonomous navigation technology with visual perception algorithms to realize automated path planning, data collection, intelligent detection and result visualization of plant health inspection. It replaces manual inspection, improves inspection efficiency, reduces maintenance costs, and realizes large-scale, routine plant health inspection. It is applicable to various scenarios such as landscaping, agricultural planting, and nursery cultivation. Attached Figure Description

[0020] Figure 1 This is a flowchart of a plant health intelligent inspection method based on mobile robots; Figure 2 The architecture diagram of the Swin-transformer method for intelligent plant health inspection based on mobile robots; Figure 3 This is an architecture diagram of a plant health intelligent inspection system based on mobile robots. Detailed Implementation

[0021] The following is in conjunction with the appendix Figure 1 To be continued Figure 3 The principles and features of the present invention are described, and the examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0022] Table 1: English-Chinese Glossary of Proper Nouns

[0023] This implementation case uses routine pathological monitoring of crabapple (tree) and purple-leaf plum (shrub) in urban park green belts as an example to verify the specific execution process of the present invention.

[0024] like Figure 1 , Figure 3 As shown, this embodiment uses a wheeled mobile robot as the inspection platform, integrating a high-resolution visible light imaging unit, an inertial measurement unit (IMU), and a global navigation satellite system (GNSS) module to carry out routine pathological monitoring in urban park green areas.

[0025] Inspection route planning: The system constructs a digital twin model of the inspection area based on the park's GIS map, sets the inspection speed to 0.5 m / s, and sets areas with high vegetation coverage density as high-priority inspection nodes to ensure the monitoring coverage of the core green landscape.

[0026] Real-time visual acquisition and pose synchronization: The robot uses SLAM technology for autonomous path planning and real-time localization. During movement, a high-resolution visible light imaging unit continuously acquires RGB images of vegetation, while RTK-GPS and IMU modules simultaneously record the robot's precise pose information (latitude, longitude, and attitude angle). All visual images and onboard pose data are synchronized with a unified timestamp, encapsulated into standardized data frames, and stored on a local edge server to ensure that the data stream (i.e., images and spatial locations captured) can be correlated based on timestamps.

[0027] Table 2: List of collected data (example of a single inspection node):

[0028] In this embodiment, the processing architecture extracts vegetation disease features from high-resolution RGB images collected by the quadrupedal mobile robot: The system uses a convolutional network module to preprocess RGB images, focusing on scanning local areas. Through a sliding window operation of the convolutional kernel, it extracts the edges of leaf lesions, mold texture, and local color anomalies caused by disease (such as yellowing and scorching). This step aims to enhance the model's sensitivity to minute pathological features on the plant surface, enabling it to quickly pinpoint abnormal texture details in the image.

[0029] like Figure 2As shown, (a) is the overall architecture of the model, which includes a local texture convolutional coding stream and a global SwinTransformer multi-scale coding stream. A hybrid feature pyramid is constructed by weighted fusion of the two streams, and a joint perceptual classifier is connected to achieve synchronous decision-making on disease type and severity level; (b) is the two-layer computation unit of the Swin Transformer block. Through the alternating mechanism of window self-attention (W-MSA) and moving window self-attention (SW-MSA), the long-range semantic association mining of disease phenotypic features between local texture and global context is realized, thereby eliminating the influence of complex garden background noise on fine-grained monitoring.

[0030] The RGB image is input into the Swin Transformer branch, where a hierarchical encoder performs multi-scale block processing. A self-attention mechanism is used to model the distribution pattern of diseases in the plant canopy, i.e., the disease evolution trend from leaves to the whole plant. This step effectively solves the recognition interference caused by complex garden backgrounds (such as alternating strong and weak light, and mutual occlusion of leaves), ensuring the model's global perception of plant health status.

[0031] Locally extracted texture features are fused with globally extracted morphological and semantic features in real time using weighted fusion. By dynamically adjusting the proportions of both in the feature space, a hybrid feature set that takes into account both "microscopic details" and "macroscopic state" is constructed. This feature set serves as the core input for subsequent disease identification and health grading, ensuring accurate diagnosis of plant diseases based solely on visible light images without relying on additional sensors.

[0032] Table 3: List of Processed Data (Example of a Single Inspection Node):

[0033] To achieve multi-dimensional situational awareness of plant diseases, this invention constructs a fine-grained labeled dataset that includes disease types, nutrient stress types, and severity levels. The specific construction process is as follows: Multi-dimensional scenario-based sampling: During different phenological periods such as spring, summer, and autumn, and covering various complex meteorological conditions including sunny days (strong light), cloudy days (weak light), and post-rain (humid), vegetation inspection data of trees and shrubs such as crabapple and purple-leaf plum in the park were collected. This sampling strategy effectively ensured the diversity of samples under complex garden backgrounds, multiple perspectives, and multiple lighting conditions.

[0034] Expert-level fine-grained annotation: Two personnel with plant pathology qualifications perform a dual annotation and cross-checking process to define the semantics of the collected samples in a fine-grained manner. Disease and stress types are labeled as follows: They are further subdivided into fungal diseases (such as rust), insect diseases (such as spider mites), bacterial diseases (such as leaf spot), and nutritional stress symptoms such as nitrogen deficiency (yellowing), iron deficiency (interveinal chlorosis), and magnesium deficiency (leaf margin scorching). Health status rating: Based on the coverage of lesions, the area of ​​leaf chlorosis, and the plant's growth status, it is uniformly divided into three severity levels: "healthy", "mild disease", and "severe disease".

[0035] Structured dataset storage: The above annotation results are semantically aligned with RGB images at the pixel level or region level to construct a fine-grained structured visual dataset containing xxx samples. This dataset is divided into training, validation, and test sets in an 8:1:1 ratio, serving as the foundational data support for feature mining and multi-task joint perceptual inference in deep learning models.

[0036] Table 4: List of Sampled Data (Full Dataset):

[0037] Table 5: List of labeled data (single sample example):

[0038] Table 6: Dataset Partitioning and Storage

[0039] Detection model training: This embodiment takes the intelligent inspection of crabapple and purple-leaf plum trees in an urban park as an example to specifically describe the execution process of the parallel coding architecture. This method specifically implements the technical solutions described in claims 1 to 6, especially through the dual-stream sensing architecture based on Swin-Transformer as described in claim 3. The specific steps are as follows: First, using the parallel processing logic of local texture flow and global semantic flow as described in claims 1 and 2, the robot, equipped with a high-resolution visible light imaging unit, acquires images of crabapple and purple-leaf plum leaves. In the local texture branch, the convolutional network performs a sliding scan of the image through convolution kernels, accurately extracting the texture density of orange-yellow spots on rust spore clusters and the gray-brown gradient information of the edges of purple-leaf plum disease. This branch serves as the texture base, ensuring that even under dappled sunlight, the model can capture the subtle texture differences between spots and normal leaves, effectively filtering out high-frequency noise interference from background weeds.

[0040] Furthermore, based on the multi-task collaborative perception branch design described in claim 3, the Swing Transformer branch performs four-level hierarchical encoding (Stage 1-4) on the whole-plant vegetation image. This branch establishes a correlation between local rust spots and the overall canopy distribution of the crabapple tree through a moving window self-attention mechanism. For example, when the model detects that rust spots exhibit a spatial structure spreading from the inside out, the Transformer branch can effectively identify the distribution pattern of the disease on the plant canopy. Similarly, for the leaf margin scorching of the purple-leaf plum, the model not only captures the local discoloration of the leaf margin but also perceives the overall decline in growth vigor caused by large-area water loss in the leaves through a global attention mechanism.

[0041] Finally, the system spatially aligns and dynamically weights the local texture features with the global semantic features. This collaborative mechanism enables the model to dynamically distinguish between different diseases: when dense texture points appear in a crabapple image and the global space exhibits a typical rust evolution structure, it is identified as "crabapple rust"; when purple-leaf plum shows localized scorched textures and overall chlorosis, it is identified as "purple-leaf plum scorch". Through this perception of "local detail guidance + global situational constraints", the system not only achieves accurate identification of disease causes during garden inspections, but also simultaneously outputs a health assessment level of "mild", "moderate", or "severe" based on the coverage of lesions and their spread on the plant.

[0042] Table 7: List of output data after grouping (single sample example);

[0043] In this embodiment, to achieve deep semantic perception of plant diseases such as crabapple rust and purple-leaf plum scorch, the system adopts a four-stage (Stage 1-4) Swin Transformer architecture for multi-scale feature encoding: after the image is input into the encoder, each stage reduces the feature map resolution step by step through the Patch Merging module. This process simulates the human eye's perception process from close-up observation of local lesions on leaves to long-distance assessment of the overall plant health status.

[0044] Hierarchical Channel Dimension Expansion: As spatial resolution decreases, the model transforms scattered pixel information into high-order pathological semantic features by progressively multiplying the feature channel dimensions. This process ensures that while compressing features, it not only preserves the salient features of lesion texture but also deeply explores the structural evolution patterns of vegetation canopy morphology. The feature maps output at each stage are channel-mapped through convolutional layers, uniformly adjusting the feature dimensions to construct an enhanced multi-scale visual feature pyramid.

[0045] Low-level features (Stage 1-2): Provide high-resolution texture support for precise localization of small spore masses of crabapple rust or initial necrotic spots of purple-leaf plum.

[0046] High-level features (Stage 3-4): Provide deep spatial correlation information to determine whether the disease has spread to the entire plant, thereby quantifying the severity level of the plant.

[0047] Through this structure, the model achieves dynamic encoding of disease phenotypes at different scales, effectively solving the problem of missed detection and misjudgment caused by different sizes of lesions in garden inspections, and providing a solid semantic foundation for subsequent fine-grained joint classification decisions. Table 8: Output characteristics of each stage of the Swing Transformer encoder:

[0048] Table 9: Final Output Data

[0049] In this embodiment, for the monitoring tasks of crabapple and purple-leaf plum leaves, the system simultaneously performs disease type identification and health level rating based on the identification results: The model first qualitatively identifies abnormal areas in plants. For example, for small orange-yellow spots on crabapple leaves, the system uses a fine-grained classification branch to extract their color characteristics (orange-yellow), texture distribution (dense spore masses), and spatial distribution (mainly on the underside of the leaf), thus accurately identifying it as "crabapple rust." For purple-leaf plum, the system captures the grayish-brown necrotic features at the leaf edges and identifies it as "leaf margin scorch." This branch directly pinpoints the causal category of the plant disease through the probability distribution output by the fully connected layer.

[0050] Based on disease identification, the system simultaneously grades the severity of the disease. For crabapple rust, the model analyzes the coverage area of ​​lesions and the degree of leaf curling to quantitatively assess it as "healthy (no disease)," "mild disease (dense spots)," or "severe disease (large-area leaf necrosis)." For purple-leaf plum, the system monitors the proportion of withered leaf area to the total leaf area and simultaneously outputs the corresponding health damage level. This process uses ordinal grading logic to output the real-time health status of the vegetation under that disease dimension.

[0051] The two perceptual branches mentioned above share visual semantic features during model training. This means that when identifying "rust disease," the model will automatically strengthen its focus on the "rust spot density" feature; and when assessing the "scorch degree," it will also conversely constrain the accuracy of the disease category. Through this joint learning strategy, the system can simultaneously generate an integrated diagnostic record of "crabapple rust - severe" or "purple-leaf plum scorch - mild" for a single RGB image during quadruped robot inspections. The trained model is deployed to a GPU server, and the robot autonomously inspects along a preset path, collecting data in real time and asynchronously inputting it into the model. It outputs scalar values ​​of plant abnormalities, pest / disease / nutrient deficiency types, and health status, displaying the vegetation growth status within the entire inspection range, and directly pushing it to the cloud management platform to guide subsequent refined spraying or pruning operations.

[0052] Table 10: Processing data (taking a 1 / 4 scale as an example):

[0053] Table 11: Output List of Enhanced Unified Feature Pyramid

[0054] Based on the enhanced unified feature pyramid, a dual-task perception branch is constructed to achieve fine-grained discrimination and quantitative assessment of plant diseases: In this embodiment, to achieve accurate quantitative perception of crabapple and purple-leaf plum, the system deploys a dual-branch collaborative perception module at the decoding end, and realizes the following perception process by performing feature decoding on the fused features: Fine-grained joint diagnosis of disease type and severity: The system inputs the identification results into the joint perception branch, and extracts a global semantic vector through a global average pooling layer (GAP). This vector is mapped to a joint probability matrix. (For example: [Crabapple rust - mild, Crabapple rust - moderate, Crabapple rust - severe]). In this case, through Softmax normalization and Argmax decision, the system can directly determine whether the crabapple has rust and its corresponding damage level from the RGB image, effectively avoiding the secondary calculation error caused by "first identifying the disease and then assessing the severity" in traditional monitoring.

[0055] Multi-dimensional quantitative rating of vegetation health status: For the three dimensions of disease, pests, and nutrient stress, the health status assessment branch utilizes a lightweight MLP to perform nonlinear regression analysis on the fused features. The model independently outputs discrete state probability distributions. (That is, the health status, mild disease level, and severe disease level of each of the following: disease, pests, and nutritional stress). This assessment branch gives the robot multi-dimensional diagnostic capabilities: for example, when the model identifies that the leaf margins of purple-leaf plum are scorched and accompanied by mild nutritional stress (potassium deficiency), it can output multi-dimensional diagnostic levels in real time, guiding maintenance personnel to carry out disease prevention and root fertilization simultaneously.

[0056] Joint Optimization Training Implementation: To ensure multi-task perception accuracy, this invention employs a collaborative training strategy. The total loss function is defined as: , When training the crabapple and purple-leaf plum datasets, we set classification weights. With hierarchical weights .

[0057] Fine-grained recognition optimization: A label-smoothed cross-entropy loss is introduced for disease classification, by setting a smoothing factor. The forced model focuses on subtle phenotypic differences between mild and severe diseases.

[0058] Health status collaborative constraints: For multi-dimensional health classification tasks, multi-class cross-entropy loss is used for supervised evaluation.

[0059] Through end-to-end joint backpropagation, the parameters of the sensing branch and the encoder backbone are optimized synchronously, enabling the model to have an integrated intelligent diagnostic capability of "qualitative diagnosis of disease and quantitative assessment of severity" for a specific tree species under a single RGB input, which significantly improves the level of intelligent perception in the inspection task of the quadruped robot.

[0060] Table 12: Loss Function Calculation Data (Single Batch Example, B=8):

[0061] Table 13: Training process data:

[0062] Table 14: Model Output Data

[0063] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent inspection of plant health based on mobile robots, characterized by: S1 Data Acquisition: A mobile robot equipped with an RGB imaging unit, an inertial measurement unit (IMU), and a global navigation satellite system (GNSS) module is deployed in a green area. The initial inspection path is set and parameters are configured based on the geographic information system (GIS) map. The robot moves autonomously through simultaneous localization and mapping (SLAM) technology, simultaneously acquiring visible light images, multispectral images, environmental metadata, and pose information. All data is stored in a standardized manner after adding a unified timestamp. S2 Data Feature Construction: The visible light image acquired in step S1 is reconstructed. The image is divided into several pixel blocks by the PatchPartition module and mapped into a multi-dimensional visual feature vector by the Patch Embedding layer. The texture edge features and global spatial distribution features of the image are extracted by preprocessing operators, and multi-scale aggregation is performed along the channel dimension to generate a feature dataset with hierarchical visual semantics. S3 Annotated Dataset Construction: Following steps S1 and S2, sampling was performed under different seasons, weather conditions, and plant growth stages to obtain a multi-time period visual feature dataset. Professionals then used a dual-annotation + cross-verification process to annotate the samples in the multi-time period visual dataset with pest and disease categories, nutrient deficiency types, and health severity levels, thus constructing a structured fine-grained labeled dataset. S4 Training of the Swin-Transformer-based dual-stream sensing model: The end-to-end multi-task plant health analysis model is trained using the structured labeled dataset described in step S3. Through the multi-scale self-attention mechanism of the Swin Transformer, the identification of abnormal vegetation areas, the classification of abnormal causes, and the grading of health status are realized. After the perception model is trained, a mobile robot performs inspections, inputting RGB images into the SwingTransformer encoder and outputting plant disease and pest categories, nutrient deficiency types, and health damage levels.

2. The intelligent plant health inspection method based on a mobile robot according to claim 1, characterized in that, Step S4 involves inputting the RGB image into the Swin Transformer encoder. First, the image is divided into 4×4 pixel blocks by the Patch Partition module, and then linearly mapped into a feature sequence by the Patch Embedding layer. ,in The feature sequence passes through four cascaded stages (Stages 1-4), each stage containing several Swin Transformer Blocks and an optional Patch Merging layer (except for Stage 4), where: The Swin Transformer Block extracts local-global features based on window multi-head self-attention (W-MSA) and sliding window multi-head self-attention (SW-MSA); The Patch Merging layer downsamples the feature map resolution to half of its original value while doubling the number of channels, thereby constructing a multi-scale, hierarchical feature pyramid. The downsampling factors were 4×, 8×, 16×, and 32×, respectively, to achieve deep semantic modeling of the subtle texture of lesions to the overall spread trend of vegetation.

3. The intelligent plant health inspection method based on a mobile robot according to claim 1, characterized in that, Step S4 describes the training process of the Swin-Transformer-based dual-stream sensing model, which includes: S4.1 Hierarchical visual semantic mapping utilizes the Patch Partition layer to map the preprocessed RGB image. Cut into , and The input image consists of a sequence of pixel blocks of height and width, respectively; each pixel block is mapped to a feature vector through a Patch Embedding layer. The feature sequence X enters a Swin Transformer encoder consisting of four cascaded stages (Stage 1-4); each stage contains several Swin Transformer Block modules, which achieve step-by-step downsampling of feature resolution through a Patch Merging layer. By progressively increasing the feature channel dimension, a multi-scale, hierarchical feature pyramid set is constructed. This pyramid structure enables deep semantic modeling from the subtle textures of local lesions to the spatial spread trend of the entire plant vegetation, providing an input basis for subsequent fine-grained disease perception. S4.2 Window Attention Feature Mining: Within each Stage, pathological features are mined using a moving window self-attention mechanism (SW-MSA). For input features... The calculation process is as follows: S4.2.1 Linear Feature Projection: Projecting the segmented Patch sequence Perform linear projection: ;in, The weight matrix is ​​a learnable matrix. For the dimension of attention head, The total number of pixels (patches) within the window; S4.2.2 Relative Position Offset and Attention Calculation: Introducing a two-dimensional relative position offset matrix. Calculate the global long-range semantic correlation degree: ;in The enhanced feature sequence, aggregated through a self-attention mechanism, represents the strength of the semantic association between patches. These are the query matrix, key matrix, and value matrix, each with a dimension of [missing information]. ; Key matrix The transpose of the matrix is ​​used to perform spatial dot product operations to capture the correlation between image patches; This is the scale scaling factor. This factor represents the dimension of the attention head; scaling using this factor can effectively prevent the value after the dot product operation from becoming too large. The function enters the gradient flat region, ensuring training stability; This is a two-dimensional relative position bias matrix; this matrix introduces the two-dimensional geometric topological relationship of plant lesions on the image surface into the feature aggregation process by encoding the relative spatial distance between patches within the window. The normalized exponential function, through calculation, maps the correlation between features to... The probability distribution weights of the intervals represent the degree of attention each Patch node pays to the current center point; This represents the total number of patches contained within the window, i.e., the length of the feature sequence. S4.3 Multi-branch Parallel Decoding and Collaborative Decision-Making: To determine the overall health status of the plant, a Global Average Pooling (GAP) operation is first performed to aggregate spatial features into a single semantic vector. : ;in, The aggregated global feature vector has a dimension of . This characterizes the overall pathological semantic state of vegetation images; For height; Width; For feature maps in spatial coordinates The eigenvector at that location; It is a spatial normalization factor. By averaging all pixels in the feature map, it achieves feature smoothing and dimensionality reduction in the spatial dimension, eliminating the influence of the specific location offset of the lesion in the image on the discrimination result. S4.3.1 Fine-grained classification branch, outputting the probability distribution of disease categories. , ;in, This represents the preset total number of disease types; The learnable weight matrix for the classification branch has dimensions of . This is used to map the global semantic feature space to the disease category decision space; Let be the bias vector for the classification branch, with dimension . This is used to adjust the output threshold for disease categories; The global semantic feature vector output by the Global Average Pooling (GAP) layer has a dimension of . This characterizes the overall health semantics of vegetation; S4.3.2 Hazard level classification branch, outputs the probability distribution of severity levels. ;in, The total number of severity levels is determined by a probability distribution. The operation realizes the quantitative judgment of vegetation health status; W1 and W2 are the weight matrices of the first and second fully connected network layers, respectively, used to learn the nonlinear mapping from the feature space to the health level space; b1 and b2 are the bias terms of the two fully connected network layers, respectively, used to adjust the activation threshold of perception decision; ReLU(⋅) is a linear rectified activation function, used to introduce nonlinear feature expression ability and enhance the model's depth of mining pathological evolution features; Softmax(⋅) is a normalized exponential function, which maps the output layer to a probability distribution vector of K levels; The S4.4 end-to-end joint optimization strategy defines the total loss function as the weighted sum of the loss from disease type identification and the loss from severity classification, and uses the backpropagation algorithm to update network parameters synchronously. Through multi-task collaborative training, the model achieves deep coupling and dynamic optimization of disease characterization and severity measurement.

4. The intelligent plant health inspection method based on a mobile robot according to claim 3, characterized in that, The dual-stream sensing branch described in step S4 is implemented by constructing a fine-grained classification branch and a health level assessment branch in parallel at the decoding end. Fine-grained classification branch: A global average pooling layer (GAP) is used for global spatial semantic aggregation. After mapping through a fully connected layer, a fine-grained joint probability distribution of disease type and severity is output through a softmax activation function. ;in, This represents the total number of disease types. This represents the severity level; this branch achieves accurate qualitative identification of the causes of diseases through a composite mapping of category attributes and evolution degree. Health level assessment branch: Employs a lightweight multilayer perceptron (MLP) architecture, independently receiving global semantic features. Through two fully connected layers and the ReLU activation function, a multi-dimensional quantitative feature vector of vegetation health status is output. Finally, a softmax layer outputs a discrete rank distribution of three dimensions: disease, pests, and nutrient stress. This branch achieves quantitative analysis of the overall health status of plants through feature decoupling.

5. The intelligent plant health inspection method based on a mobile robot according to claim 3, characterized in that, The end-to-end joint optimization strategy described in step S4.4 includes: S4.4.1 Constructing the composite total loss function Joint and collaborative optimization of multi-task-aware branches; ， in, The weighting coefficients are determined through a grid search on the validation set to balance them. S4.4.2 Classification of Disease Types and Losses The label-smooth cross-entropy loss is used, as shown in the following formula: ， in, For the total sample size, This represents the total number of preset disease categories; For the smoothed true label, These are the probability values ​​for the types of diseases predicted by the model. 4.4.3 Severity Classification of Losses Multi-class cross-entropy loss is used to optimize the quantitative perception of vegetation health status. ， in, The total number of preset severity levels, For the total sample size, For the corresponding severity level, use one-hot encoded real labels. The predicted rank probability value by the model; The collaborative optimization mechanism synchronously updates the weight parameters of the encoder and the decoding sensor head through the backpropagation algorithm, enabling the model to simultaneously possess the ability to distinguish disease types and quantitatively assess health levels in a single RGB mode, effectively enhancing the comprehensive representation accuracy of the feature backbone for vegetation pathological evolution characteristics.

6. The intelligent plant health inspection method based on a mobile robot according to claim 1, characterized in that: The environmental metadata in step S1 includes temperature, humidity, and light intensity; the effective observation distance of the multispectral image unit is 0.5–3 meters; the configuration parameters include inspection speed, detection confidence threshold, and task priority strategy based on vegetation density.

7. The intelligent plant health inspection method based on a mobile robot according to claim 1, characterized in that, In step S3: the pest and disease categories include fungal diseases, insect pests, and bacterial diseases; the nutrient deficiency types include nitrogen deficiency, iron deficiency, and magnesium deficiency; and the health severity levels include normal, mild disease malnutrition, and severe disease malnutrition.

8. A plant health intelligent inspection system based on mobile robots, characterized in that, The method for implementing any one of claims 1 to 7 includes a hardware integration module and a software algorithm module; The hardware integration module is a mobile robot platform equipped with multiple sensors, including the mobile robot body, a visible light imaging unit, an IMU module, a GNSS module, and an autonomous navigation module integrated into the body. The visible light imaging unit is used to acquire RGB images of vegetation in real time during the inspection process, and the autonomous navigation and SLAM module is used to plan the inspection path and locate in real time. The software algorithm module includes a data acquisition module, an encoding module, and a visual-semantic collaborative perception module. The data acquisition module controls the hardware integration module to acquire visible light images in real time and simultaneously add pose and timestamp information. The encoding module uses Swing Transformer to extract vegetation texture and semantic features. The visual-semantic collaborative perception module constructs a multi-task joint perception network based on hybrid features to achieve integrated identification and quantitative assessment of vegetation disease types and health severity, and outputs diagnostic results through a visualization terminal.

9. The intelligent plant health inspection system based on a mobile robot according to claim 8, characterized in that: The encoding module has a built-in multi-task decoding head and a loss calculation unit. The multi-task decoding head includes a detection head, a segmentation head, and a regression head, which respectively realize the location and classification of abnormal areas, the precise location of diseased areas, and the assessment of health status. The loss calculation unit has built-in detection loss, nutrient status regression loss, and total loss functions, and supports determining the loss weight through validation set grid search and dynamically fine-tuning it according to disease type.

Images