Lawn health state recognition method, device and equipment, medium and management system

Abstract

The invention relates to the technical field of lawn maintenance, and discloses a lawn health state recognition method, device and equipment, a medium and a management system. A lawn health state recognition model based on multi-modal data fusion (RGB images, elevation maps, vegetation indexes and vibration data) is constructed; and efficient online reasoning, intelligent data management and closed-loop operation control are integrated, so that the recognition precision and robustness of the lawn health state recognition model on lawn health and disease types in a complex scene can be remarkably improved, storage and communication resources of embedded equipment are greatly optimized through differentiated data management strategies, and the reliability of the lawn health state recognition model is improved. Real-time dynamic linkage of an identification result, a lawn mower map and an operation strategy is realized, a complete closed loop from accurate perception to intelligent maintenance is formed, and efficient, accurate and resource-saving lawn health state autonomous maintenance is achieved.

Description

Technical Field

[0001] This invention relates to the field of lawn maintenance technology, and in particular to a method, device, equipment, medium and management system for identifying the health status of lawns. Background Technology

[0002] With advancements in smart horticulture and landscape maintenance technologies, smart lawnmowers have become increasingly common, their functions no longer limited to basic lawn trimming. Users have higher expectations for their ability to monitor lawn health and provide precise maintenance. Meanwhile, in the field of smart agriculture, the automated identification and diagnosis of vegetation health using computer vision and machine learning technologies has become an important research direction. However, when these technologies are specifically applied to lawn maintenance scenarios, existing solutions still have a series of significant limitations in model building, data processing, and system closed-loop mechanisms.

[0003] First, existing methods for constructing lawn health status recognition models have significant shortcomings. Most current research and methods rely primarily on single RGB image data for model training. This reliance on a single appearance feature makes it difficult for models to distinguish lawn problems with similar visual features but drastically different causes. For example, an area exhibiting a withered, yellowish color could be caused by drought, pests and diseases, nutrient deficiency, or fungal infection; relying solely on RGB image information easily leads to misjudgment. Furthermore, existing models generally lack the ability to perceive the three-dimensional geometry and physical properties of lawns, failing to effectively acquire and understand crucial information such as potholes, subsurface thickness, and soil compaction. This information is key to diagnosing latent problems such as "waterlogging" and "soil compaction" that are not directly visible on the surface. Moreover, models trained on single-modal data often exhibit poor generalization ability and robustness. Their performance fluctuates drastically under different grass species (e.g., cool-season vs. warm-season grasses), different light conditions, and different seasonal variations, making stable application difficult. This problem is further exacerbated by the high cost and scarcity of accurately labeled lawn disease data, which restricts the development of data-driven models.

[0004] Secondly, existing methods also face significant challenges in the deployment and systematic application of recognition models. Deploying recognition models on the embedded platform of a smart lawnmower, where computing resources, storage space, and power consumption are extremely limited, and achieving real-time, efficient online inference, is a major engineering challenge. Furthermore, smart lawnmowers continuously generate massive amounts of image and environmental sensor data during daily operations. Storing or uploading all raw data indiscriminately would quickly exhaust the device's local storage space and incur high wireless communication costs. Moreover, most of this data actually comes from invalid or redundant information from healthy lawn areas, resulting in a huge waste of storage and bandwidth. On the other hand, existing health recognition methods often stop at the "recognition" stage itself; the identified problem area information fails to form an effective intelligent closed loop with the lawnmower's core operation control unit (such as path planning and operation execution mechanisms). This prevents the recognition results from being directly translated into precise maintenance actions (such as targeted fertilization, watering, or avoidance), causing a disconnect between the "recognition" and "maintenance" stages. Meanwhile, the identified changes in lawn health were not dynamically and intelligently integrated and updated with the lawnmower's internal navigation map, resulting in map information lagging significantly behind the actual health condition of the lawn. As a result, the lawnmower may repeatedly run over identified vulnerable areas, exacerbating the damage, or continue to perform unnecessary special operating strategies in recovered healthy areas.

[0005] In summary, existing methods for identifying lawn health status have several shortcomings: First, at the model level, there is a lack of health status identification models that can comprehensively utilize multi-dimensional information and possess strong generalization capabilities and modality-deficient adaptability. Second, at the system level, there is a lack of a complete online identification and response process that integrates efficient online inference, intelligent data filtering, real-time map updates, and adaptive operation adjustments. These shortcomings collectively limit the progress of intelligent lawnmowers in developing into intelligent and precise lawn health maintenance experts. Summary of the Invention

[0006] This invention provides a method, apparatus, equipment, medium, and management system for identifying the health status of lawns, in order to overcome the deficiencies of the prior art.

[0007] On one hand, the present invention provides a method for identifying the health status of a lawn, comprising: Acquire raw sensing data of the lawn area to be treated, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; Based on the original sensing data, the lawn health status identification result of the lawn area to be treated is obtained by using a preset lawn health status identification model. The lawn health status identification result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

[0008] According to the lawn health status identification method provided by the present invention, before acquiring the original sensing data of the lawn area to be treated, the method further includes: Construct the preset lawn health status recognition model; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0009] According to the lawn health status identification method provided by the present invention, the multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0010] According to the lawn health status identification method provided by the present invention, the progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

[0011] According to the lawn health status identification method provided by the present invention, the adversarial modality discard training method refers to randomly discarding any number of training data of any number of modalities during the training process of the lawn health status identification model, so as to simulate the situation of data missing and / or lost.

[0012] The lawn health status identification method provided by the present invention further includes: Obtain the current map status information of the lawn area to be treated from the lawn health semantic map; The lawn health status identification results are compared with the map status information, and differentiated data management and lawn health semantic map management are performed based on the comparison results.

[0013] According to the lawn health status identification method provided by the present invention, the step of comparing the lawn health status identification result with the map status information, and performing differentiated data management and lawn health semantic map management based on the comparison result, includes: When the confidence level of the prediction result is lower than the preset threshold, a conservative strategy is implemented. If the confidence level of the prediction result is equal to or greater than a preset threshold, it is determined whether the lawn health status identification result is consistent with the map status information. If they are consistent, a storage-saving management strategy is adopted; if they are inconsistent, a knowledge-updating management strategy is adopted.

[0014] The lawn health status identification method provided by the present invention further includes: Based on the lawn health status identification results, the operation strategy for the lawn area to be worked on is adjusted.

[0015] On the other hand, a lawn health status identification device is provided, the device comprising: The status detection module is used to acquire the raw sensing data of the lawn area to be worked on, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; The status recognition module is used to obtain the lawn health status recognition result of the lawn area to be treated based on the original sensing data and through a preset lawn health status recognition model. The lawn health status recognition result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

[0016] On another front, an autonomous mobile device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the lawn health status identification method described above.

[0017] In another aspect, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, wherein when the processor executes the program, it implements the lawn health status identification method as described above.

[0018] In another aspect, a computer program product is provided, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the above-described lawn health status identification method.

[0019] On another front, a lawn health status management system is provided, including the autonomous mobile device as described above, and one or any combination of a cloud server, a user terminal, a data processing device, and an operation device. The cloud server or user terminal is used to receive and display the lawn health status identification results and / or the real-time updated lawn health semantic map sent by the autonomous mobile device. The data processing device is used to receive and perform differentiated data management and lawn health semantic map management based on the comparison results between the lawn health status identification results and the map status information. The operation device is used to receive and perform lawn operations based on the real-time adjusted operation strategy.

[0020] The present invention provides a method, apparatus, equipment, medium, and management system for identifying the health status of lawns, which can bring at least the following beneficial effects: (1) In terms of model construction and recognition accuracy: This invention integrates four modal information—RGB images (appearance), elevation maps (geometry), vegetation indices (spectral), and vibration data (physical)—to create a lawn health status identification model. This model comprehensively utilizes the appearance, structure, physiological, and physical characteristics of lawns, fundamentally overcoming the problem that a single RGB image model cannot distinguish between visually similar but differently caused conditions (such as drought, pests, and nutrient deficiency). This enables effective diagnosis of hidden problems such as "groundwater accumulation" and "soil compaction."

[0021] This invention employs a multi-branch neural network architecture and a progressive training strategy, enabling the lawn health status recognition model to fully learn the stable characteristics of different modalities under various scenarios. In particular, through adversarial modality dropout training, the model is forced to adapt to the actual situation of missing sensor data, ensuring that the lawn health status recognition model maintains stable and reliable performance in complex real-world environments (such as different grass species, light intensity, and seasons) and when facing equipment failures or data quality issues.

[0022] The lawn health status recognition model employs a multi-granularity classifier (distinguishing between "healthy / unhealthy" and specific problem types) and a progressive training method, which greatly reduces the reliance on a large amount of finely labeled data and improves the model's usability in scenarios with limited labeled data.

[0023] (2) Regarding embedded deployment and resource utilization: The lawn health status recognition model itself adopts a lightweight branch design and dynamically adjusts the calculation path through an adaptive branch weight learning module. While ensuring accuracy, it minimizes computational overhead, enabling complex high-performance multimodal fusion models to run in real time on embedded platforms such as smart lawnmowers.

[0024] This invention intelligently compares the identification results with the map status and executes differentiated data management strategies based on the comparison results and prediction confidence. It proactively discards redundant data indicating "health" or "problem persistence," retaining only key evidence such as "newly discovered problems" or "problem recovery" that signify status changes. This fundamentally solves the problems of storage space exhaustion and high communication costs caused by massive amounts of data.

[0025] (3) In terms of system closed-loop and precision maintenance: This invention directly links the identification results with the operation control unit, enabling adaptive adjustment of operation strategies (such as avoiding problem areas, adjusting mowing height, and triggering targeted pesticide application) based on the specific problem type identified (such as brown spot disease and groundwater accumulation), forming an intelligent closed loop of "perception-decision-action", upgrading from passive identification to proactive maintenance.

[0026] This invention achieves real-time and accurate map updates of lawn health status by dynamically integrating the recognition results with the internal navigation map. This enables the lawnmower to plan its path based on the latest map information, avoiding repeated trampling of vulnerable problem areas or ignoring restored areas, thus achieving intelligent and humanized lawn maintenance.

[0027] In summary, this invention, through end-to-end innovation from the construction of a lawn health status recognition model (multimodal fusion, progressive training) to online inference (adaptive weights, lightweighting), and then to system application (intelligent data filtering, real-time map updates, and closed-loop operation control), constructs a complete intelligent method capable of autonomous health diagnosis and precise maintenance. This can greatly promote the development of intelligent lawnmowers into professional and intelligent lawn health maintenance experts. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a method for identifying the health status of a lawn provided by the present invention.

[0030] Figure 2 This is a schematic diagram of the structure of a lawn health status identification device provided by the present invention.

[0031] Figure 3 This is a schematic diagram of the structure of the autonomous mobile device provided by the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0033] The present invention provides a method for identifying the health status of lawns, including the construction and application of a lawn health status identification model. Figure 1This is a flowchart illustrating a lawn health status identification method provided by the present invention. The execution subject of this lawn health status identification method can be any autonomous mobile device suitable for lawn operations, such as an autonomous lawnmower, automatic aerator, autonomous grass planter, or autonomous weed cutter. The autonomous mobile device can acquire actual data while operating using its own vision module (RGB camera or multispectral imaging system) and motion sensing module (inertial measurement unit, IMU), thereby achieving lawn health status identification.

[0034] See Figure 1 The present invention provides a method for identifying the health status of a lawn, the method comprising: S10. Construct a preset lawn health status recognition model.

[0035] Step S10 includes the following sub-steps: Sub-step S110: Acquire historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels for the lawn area. The health status labels include coarse-grained health status labels and problem type labels. Coarse-grained health status includes healthy and unhealthy, and problem types include, but are not limited to, drought, surface waterlogging, groundwater accumulation, lack of vegetation, weeds, need for weed removal, diseases, and pests. Historical elevation patches are generated using binocular vision, LiDAR, or SLAM algorithms on the autonomous mobile device. Historical vegetation index patches are calculated using the Normalized Differential Vegetation Index (NDVI) and the Normalized Differential Water Index (NDWI). Historical vibration data is obtained using the IMU sensor on the autonomous mobile device. NDVI is the most classic and commonly used vegetation index, calculated using reflectance in the near-infrared and red bands. It is mainly used to monitor vegetation growth status, biomass, and chlorophyll content; a higher value generally indicates more lush and healthier vegetation. Normalized Differential Water Index: Using the green and near-infrared bands, it is mainly used for water body identification and extraction.

[0036] After acquiring historical data, sub-step S110 also needs to preprocess the data, including normalization, data augmentation, and spatial alignment.

[0037] Sub-step S120: Based on the historical RGB image blocks, historical elevation blocks, historical vegetation index blocks, historical vibration data, and the health status labels, using a multi-branch neural network architecture, a progressive training strategy, and an adversarial mode dropout training method, the model is trained to learn the feature changes of the historical RGB image blocks, historical elevation blocks, historical vegetation index blocks, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0038] In one embodiment, the multi-branch neural network architecture includes: Visual Branch: Input: RGB three-channel image patch, for example, 128×128 pixels; backbone network: lightweight CNN (such as MobileNetV2), multi-scale feature pyramid (FPN). Visual feature maps of different levels (shallow features (high resolution, rich details), mid-level features, and deep features (high semantic information)) are extracted from the RGB image blocks. The number of channels is unified by 1×1 convolution, and then visual features are fused by upsampling and element-wise addition to obtain a multi-scale visual feature vector Fv.

[0039] Geometric branches: Input: Single-channel elevation map patch, size aligned with RGB image patch; Structure: Lightweight FPN structure similar to the vision branch, but with the number of input channels adjusted to 1. This is used to extract geometric feature maps of different levels from the elevation map blocks and perform geometric feature fusion to obtain a multi-scale geometric feature vector Fg.

[0040] Spectral branches: Input: Single-channel vegetation index tile, such as NDVI, with dimensions aligned to the above tile; Structure: Lightweight FPN structure, with 1 input channel. This is used to extract spectral feature maps of different levels from the vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector Fs.

[0041] Physics branch: Input: A low-dimensional feature vector containing vibrational features extracted from IMU data, such as dominant frequency F_curr, low-frequency energy ratio R_low, and sample entropy SampEn. Structure: Two fully connected layers, with intermediate layers using... Activation function; This is used to extract multiple vibration features from the real-time vibration data and perform vibration feature fusion to obtain a physical feature vector Fp.

[0042] The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch.

[0043] Specifically, the purpose of the adaptive branch weight learning module is to dynamically adjust the importance of each modality in the fusion based on the input samples, and adapt to changes in modality credibility under different scenarios.

[0044] Structure: Branch confidence estimation: Each branch output is followed by a lightweight confidence prediction subnetwork (2-layer MLP, 64 hidden layer dimensions), which outputs the confidence score cv, cg, cs, cp of that branch on the current sample, with values ​​ranging from [0,1].

[0045] Adaptive weight calculation formula:

[0046] in This represents the adaptive weight of the i-th branch. Assign a confidence score to this branch. The temperature coefficient hyperparameter is used to control the smoothness of the weight distribution. T controls the smoothness of the weight distribution; the larger T is, the smoother the distribution. The initial value of T can be set to 1.0, T=2.0 for the first 20 rounds (to encourage exploration), and then linearly decay to T=0.5 (to encourage focus). T is used to prevent some modality weights from converging to 0 too early in the early stages of training.

[0047] Weighted feature fusion preprocessing expression:

[0048] in, This represents the feature vector of the i-th branch after weighted processing. This is the original feature vector before processing.

[0049] Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector.

[0050] Specifically, the purpose of the cross-modal attention fusion network is to achieve dynamic interaction and deep fusion between multimodal features, and to assign different weights to different modalities based on the backpropagation of algorithm results, thereby improving the model's ability to discriminate complex lawn health conditions.

[0051] Structure: Feature Alignment and Projection: The modality-weighted features are projected onto the same latent space dimension through a fully connected layer.

[0052]

[0053]

[0054]

[0055] Cross-modal attention computation: For each modality pair (i,j), compute query-key attention:

[0056]

[0057]

[0058] in For the enhanced i-th modal feature, For the original projection features, These are query, key, and value vectors, respectively. For a learnable projection matrix, The dimension of the key vector.

[0059] Hierarchical feature fusion: The enhanced modal features are concatenated, and the concatenated vector of modal features after cross-modal attention enhancement is obtained. :

[0060] in, The visual, geometric, spectral, and physical modal features are respectively enhanced with cross-modal attention.

[0061] Dimensionality reduction and fusion through a fully connected layer:

[0062]

[0063]

[0064] A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0065] Specifically, multi-granularity classifiers include: Main classifier: Outputs coarse-grained health status (healthy / unhealthy); Fine-grained classifier: Outputs the specific problem type (groundwater accumulation / brown spot disease / fertilizer deficiency, etc.); Confidence output layer: Outputs the confidence score for each prediction result.

[0066] In one embodiment, the progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch using only one of the four modalities: visual, geometric, spectral, and physical (goal: to enable each branch to have basic feature extraction capabilities; training cycle: 50 rounds of training for each branch separately). Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical (using a simple feature concatenation fusion method; training cycle: 30 rounds for each combination). Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training (training cycle: 100 rounds of full training). Multi-granularity classification training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—a multi-task loss function is used to train the main classifier, fine-granular classifier, and confidence output layer. The main classifier outputs whether the lawn area to be treated is healthy; the fine-granular classifier outputs the types of problems existing in the lawn area; and the confidence output layer outputs the model's confidence in the prediction results. The multi-task loss function is: in, For coarse-grained classification loss weights, For fine-grained classification loss weights, α + β + γ = 1.0 are the auxiliary loss weights for branch confidence prediction, and α + β + γ are the weight adjustment coefficients.

[0067] In one embodiment, the adversarial modality dropout training method refers to randomly dropping any number of training data of any number of modalities during the training process of the lawn health status recognition model, in order to simulate the situation of data missing and / or lost. The dropout probability increases linearly with the number of training rounds, from 10% to 40%, forcing the model to learn to make reliable predictions even when modalities are incomplete.

[0068] The following examples demonstrate the performance of the lawn health status recognition model.

[0069] Example 1: Validation of the training and expansion capabilities of a standard multimodal model Scenario: Build a lawn health status recognition model for a golf course maintenance system.

[0070] Process: Using a smart lawnmower equipped with a multispectral camera and RTK-GPS, 10,000 sets of multimodal data were collected, and a lawn health status recognition model was trained according to the S110-S120 process.

[0071] Comparison: Compared with existing models, the lawn health status identification model has an accuracy increase of more than 8% under the same data missing conditions.

[0072] Example 2: Generalization ability test across turf types Procedure: The model was trained on a cool-season grass (Frigida spp.) dataset and evaluated directly on a warm-season grass (Bermudagrass) test set.

[0073] Results: The accuracy was significantly higher than that of the single-scale model.

[0074] Analysis: The lawn health status identification model can capture the common characteristics of different grass species.

[0075] Example 3: Performance stability of multi-resolution input Procedure: The input image resolution was varied from 64×64 to 256×256 to test the performance fluctuation of the model at different resolutions.

[0076] Results: The performance fluctuations are small, demonstrating the effectiveness of the multi-scale FPN structure in the lawn health status identification model.

[0077] Example 4: Comparison of Progressive Training Effects Process: Comparing progressive training and end-to-end training Results: Progressive training converges faster, achieves higher final accuracy, and significantly improves training stability.

[0078] S11. Obtain the original sensing data of the lawn area to be treated, wherein the original sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data.

[0079] S12. Based on the original sensing data, the lawn health status identification result of the lawn area to be treated is obtained through the lawn health status identification model. The lawn health status identification result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

[0080] S13. Obtain the current map status information of the lawn area to be treated in the lawn health semantic map, compare the lawn health status identification result with the map status information, and perform differentiated data management and lawn health semantic map management based on the comparison result.

[0081] In one embodiment, the lawn health semantic map may be pre-built and include multiple layers, each containing various types of information, such as location information, health problem identifiers, etc.

[0082] In one embodiment, S13 may determine the management strategy according to the following rules: When the confidence level of the prediction result is lower than the preset threshold, a conservative strategy is implemented. If the confidence level of the prediction result is equal to or greater than a preset threshold, it is determined whether the lawn health status identification result is consistent with the map status information. If they are consistent, a storage-saving management strategy is adopted; if they are inconsistent, a knowledge-updating management strategy is adopted.

[0083] In one embodiment, the management strategy can be determined based on Table 1.

[0084] Table 1

[0085] S14. Adjust the operation strategy for the lawn area to be worked on based on the lawn health status identification results. In one embodiment, S14 can adjust the operation strategy according to the following rules: If vulnerable areas such as "groundwater accumulation" or "severe bald patches" are identified, instructions can be issued to control the lawnmower to detour or increase the mowing height to avoid further damage.

[0086] If an area with "overgrown weeds" is identified, the number of times that area is traversed can be increased in the path planning to strengthen weed control.

[0087] If a "damaged" area is identified, a command can be issued to slow down the lawnmower and take higher-resolution photos for further analysis.

[0088] The following examples illustrate the process of identifying lawn health status.

[0089] Example 1: Intelligent daily operation of a home lawnmower Process: The lawnmower is operating in Zhang San's backyard. Most of the time, the model judges it as "healthy," executes decision A, does not save data, and quietly mows the lawn. When it reaches a low-lying area near the corner, the model identifies it as "groundwater accumulation" (a new problem), executes decision C, saves the data, and marks it as a red restricted area on the map. The lawnmower automatically avoids the area and sends a notification to Zhang San on the app: "Groundwater accumulation detected, automatically avoided; it is recommended to check the drainage."

[0090] Example 2: Verification of restoration of abnormal areas in lawns Process: On a golf course green, a "brown spot" area was marked on the map last week. This week, when the lawnmower passed over the area again, the model identified it as "healthy." The system executed decision B, saving the recovery evidence and removing the disease marker from the map. This provided data support for assessing the effectiveness of the pesticides and avoided unnecessary continued isolation of the area.

[0091] Example 3: Efficient Data Management of Large-Scale Lawns Process: A lawnmower in a park covers a huge area with each operation. Using this method, although over 100,000 images were processed in an 8-hour operation, only about 50 newly discovered anomalous samples (Decision C) and a few "pending confirmation" samples (Decision E) were ultimately stored because 99% of the area was healthy (Decision A). Storage and communication loads were reduced by two orders of magnitude, making routine monitoring feasible.

[0092] Example 4: Model optimization loop closure based on low-confidence samples Process: A lawnmower encounters a patch of grass with an unusual color. The model identifies it as "suspected pest," but the confidence level is only 0.55 (below the threshold of 0.7). The system executes decision E, marking it as "pending confirmation" and uploading it to the cloud. After reviewing the sample, horticultural experts confirm it is a new type of pest and relabel the sample. This newly labeled sample is automatically added to the training set for incremental learning in the next round of model training, making the system increasingly intelligent.

[0093] Example 5: Multi-device collaborative operation Process: The lawnmower identifies a large area with insufficient grass (Decision C) and uploads the information to the cloud management platform. Based on the "task-device mapping table," the platform automatically generates a "replanting" task, plans the optimal path, and assigns it to an automated seeding robot. After receiving the task, the seeding robot precisely moves to the area to perform the replanting operation. Upon completion, the lawnmower verifies the restoration status during subsequent inspections (Decision B), forming a complete maintenance loop.

[0094] The lawn health status identification method provided by this invention has at least the following beneficial effects: In terms of model construction and recognition accuracy, by fusing appearance, geometry, spectral, and physical multimodal information, and combining multi-branch networks and cross-modal attention fusion, the model achieves accurate fusion of multi-source information, significantly improving the classification accuracy of the lawn health status recognition model, with an overall accuracy rate exceeding 97%. This lawn health status recognition model possesses powerful multi-scale perception capabilities, simultaneously capturing microscopic lesions and macroscopic growth trends. Through progressive training and adversarial modality discarding, not only is training efficiency and stability improved, but the model also possesses strong modal robustness, with performance degradation controlled to within 4% when some modalities are missing. The adaptive decision-making mechanism dynamically adjusts the importance of each modality, and combined with multi-scale feature extraction, it forges the model's excellent cross-scene generalization ability, demonstrating stable performance under different lawn types, seasons, and lighting conditions.

[0095] In terms of embedded deployment and resource utilization, this invention demonstrates strong engineering practicality and extremely high resource utilization efficiency. The model adopts a modular and lightweight design and achieves efficient inference through adaptive computation paths. More importantly, through the intelligent decision-making logic of "not saving health data and not redundantly recording recurring problems," the system greatly saves device storage and communication bandwidth, making low-cost, high-frequency routine monitoring a reality.

[0096] In terms of system closed-loop and precision maintenance, this invention truly achieves a complete closed loop of "perception-decision-action," directly converting AI recognition results into equipment control commands (such as detour and deceleration), upgrading the lawnmower from a "passive tool" to an "active maintenance intelligent agent." By dynamically fusing recognition results with navigation maps, dynamic and realistic map maintenance is achieved, making the map a "living map" that reflects the latest health status of the lawn, providing a precise digital foundation for all maintenance activities. Furthermore, by collecting uncertain samples and recovery evidence, a continuous self-optimizing data stream is formed, creating an intelligent system that constantly improves itself.

[0097] The lawn health status identification device provided by the present invention is described below. The lawn health status identification system described below can be referred to in correspondence with the lawn health status identification method described above.

[0098] On the one hand, see Figure 2 The present invention provides a lawn health status identification device, which may include: The status detection module is used to acquire the raw sensing data of the lawn area to be worked on, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; The status recognition module is used to obtain the lawn health status recognition result of the lawn area to be treated based on the original sensing data and through a preset lawn health status recognition model. The lawn health status recognition result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

[0099] Furthermore, the device also includes a model building module for building the preset lawn health status recognition model; specifically, it acquires historical RGB image blocks, historical elevation blocks, historical vegetation index blocks, historical vibration data, and health status labels of the lawn area, wherein the health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0100] Furthermore, the multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0101] Furthermore, the progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

[0102] Furthermore, the adversarial modality dropout training method refers to randomly dropping any number of training data for any number of modalities during the training process of the lawn health status recognition model, in order to simulate the situation of data missing and / or lost.

[0103] Furthermore, the map data acquisition module is used to acquire the current map status information of the lawn area to be treated in the lawn health semantic map; The data management module is used to compare the lawn health status identification results with the map status information, and to perform differentiated data management and lawn health semantic map management based on the comparison results.

[0104] In one embodiment, the data management module is specifically used to execute a conservative strategy when the confidence level of the prediction result is lower than a preset threshold. If the confidence level of the prediction result is equal to or greater than a preset threshold, it is determined whether the lawn health status identification result is consistent with the map status information. If they are consistent, a storage-saving management strategy is adopted; if they are inconsistent, a knowledge-updating management strategy is adopted.

[0105] Furthermore, the device also includes: The operation strategy adjustment module is used to adjust the operation strategy of the lawn area to be operated based on the lawn health status identification results.

[0106] Specific limitations regarding the lawn health status identification device can be found in the limitations of the lawn health status identification method described above, and will not be repeated here. Each module in the aforementioned lawn health status identification device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the autonomous mobile device in hardware form or independently of it, or stored in the memory of the autonomous mobile device in software form, so that the processor can call and execute the corresponding operations of each module.

[0107] The present invention also provides an autonomous mobile device, including a visual perception module, a geometric perception module, a spectral perception module, a physical perception module, a computing and control unit, and a communication module. The visual perception module is used to acquire RGB image blocks of a lawn area; the geometric perception module is used to acquire elevation maps of the autonomous mobile device in the lawn area; the spectral perception module is used to acquire vegetation index maps of the autonomous mobile device in the lawn area; the physical perception module is used to acquire real-time vibration data of the autonomous mobile device in the lawn area; and the computing and control unit is used to receive the RGB image blocks, elevation maps, vegetation index maps, and real-time vibration data, and execute the lawn health status identification method described above.

[0108] This invention also provides a lawn health status recognition system, including an autonomous mobile device as described above, and one or any combination of a cloud server, a user terminal, a data processing device, and an operation device. The cloud server or user terminal is used to receive and display the lawn health status recognition results and / or the real-time updated lawn health semantic map sent by the autonomous mobile device. The data processing device is used to receive and perform differentiated data management and lawn health semantic map management based on the comparison results between the lawn health status recognition results and the map status information. The operation device is used to receive and perform lawn operations according to the real-time adjusted operation strategy.

[0109] Figure 3 An example is a schematic diagram of the physical structure of an autonomous mobile device, such as... Figure 3 As shown, the autonomous mobile device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logical instructions in the memory 830 to execute the steps of the lawn health status identification method and / or the lawn health status identification method described above.

[0110] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to perform the steps of the lawn health status identification method and / or the lawn health status identification method described above.

[0112] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the lawn health status identification method and / or the lawn health status identification method described in any of the preceding claims.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0114] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying the health status of a lawn, characterized in that, include: Acquire raw sensing data of the lawn area to be treated, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; Based on the original sensing data, the lawn health status identification result of the lawn area to be treated is obtained by using a preset lawn health status identification model. The lawn health status identification result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

2. The method for identifying lawn health status according to claim 1, characterized in that, Before acquiring the raw sensor data of the lawn area to be treated, the method further includes: Construct the preset lawn health status recognition model; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

3. The method for identifying lawn health status according to claim 2, characterized in that, The multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

4. The method for identifying lawn health status according to claim 2, characterized in that, The progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

5. The method for identifying lawn health status according to any one of claims 2-4, characterized in that, The adversarial modality dropout training method refers to randomly dropping any number of training data for any number of modalities during the training process of the lawn health status recognition model, in order to simulate the situation of data missing and / or lost.

6. The method for identifying lawn health status according to claim 1, characterized in that, The method further includes: Obtain the current map status information of the lawn area to be treated from the lawn health semantic map; The lawn health status identification results are compared with the map status information, and differentiated data management and lawn health semantic map management are performed based on the comparison results.

7. The lawn health status identification method according to claim 6, characterized in that, The step of comparing the lawn health status identification result with the map status information, and performing differentiated data management and lawn health semantic map management based on the comparison result, includes: When the confidence level of the prediction result is lower than the preset threshold, a conservative strategy is implemented. If the confidence level of the prediction result is equal to or greater than a preset threshold, it is determined whether the lawn health status identification result is consistent with the map status information. If they are consistent, a storage-saving management strategy is adopted; if they are inconsistent, a knowledge-updating management strategy is adopted.

8. The method for identifying lawn health status according to claim 6, characterized in that, The method further includes: Based on the lawn health status identification results, adjust the operation strategy for the lawn area to be worked on.

9. A lawn health status identification device, characterized in that, include: The status detection module is used to acquire the raw sensing data of the lawn area to be worked on, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; The status recognition module is used to obtain the lawn health status recognition result of the lawn area to be treated based on the original sensing data and through a preset lawn health status recognition model. The lawn health status recognition result includes whether the lawn area to be treated is healthy, the type of problem existing in the lawn area to be treated, and the confidence level of the prediction result.

10. An autonomous mobile device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the lawn health status identification method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the processor executes the program, it implements the lawn health status identification method as described in any one of claims 1 to 8.

12. A computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of performing the lawn health status identification method according to any one of claims 1 to 8.

13. A lawn health status management system, characterized in that, The system includes the autonomous mobile device as described in claim 10, and one or any combination of a cloud server, a user terminal, a data processing device, and an operation device. The cloud server or user terminal is used to receive and display the lawn health status identification results and / or the real-time updated lawn health semantic map sent by the autonomous mobile device. The data processing device is used to receive and perform differentiated data management and lawn health semantic map management based on the comparison results between the lawn health status identification results and the map status information. The operation device is used to receive and perform lawn operations according to the real-time adjusted operation strategy.

Images