Lawn health semantic map construction method, device, equipment, medium and system

Abstract

The invention relates to the technical field of smart gardens, and discloses a lawn health semantic map construction method, device and equipment, a medium and a management system.The method comprises the steps that a multi-layer raster data stack containing a geographic layer and a semantic layer is created for a target lawn area; obtaining multi-modal sensing data of the to-be-detected area, and obtaining an identification result containing a problem type, confidence and a position through a preset lawn health state identification model; and mapping the identification result to a corresponding grid unit of the data stack, updating layer information of the grid unit according to a preset rule, and generating a dynamic lawn health semantic map. According to the method, multi-modal fusion recognition is combined with a multi-layer raster data stack, a traditional navigation map is upgraded into a dynamic digital model rich in healthy semantics, and the problems that an existing map is single in information dimension, lack of semantics and disjunction of data and application are effectively solved; and a unified data base and a decision basis are provided for realizing precise and automatic maintenance of the lawn.

Description

Technical Field

[0001] This invention relates to the field of smart garden technology, and in particular to a method, apparatus, equipment, medium and management system for constructing a semantic map of lawn health. Background Technology

[0002] In fields such as smart landscaping, precision agriculture, and home lawn care, maps are the core foundation for path planning, task management, and decision-making. Currently, the map types widely used in autonomous mobile devices such as smart lawnmowers and inspection robots are mainly occupancy grid maps or path maps. These traditional mapping technologies primarily serve the basic navigation and obstacle avoidance functions of these devices. Occupancy grid maps typically divide the environment into grid cells, with each cell containing only two states: "occupied" (impassable) or "free" (passable); while path maps mainly record the network of paths that the device can traverse.

[0003] However, with the increasing demand for refined and intelligent lawn maintenance, the aforementioned traditional mapping technologies have gradually revealed the following inherent defects and limitations: Existing maps are essentially geometric or topological maps, capable of representing only the physical structure of the environment (such as navigable areas and obstacles), but unable to convey any semantic content related to lawn health. Users cannot directly obtain key maintenance information such as "where and what problems exist" from the map, resulting in an information gap between map perception and maintenance decisions. Furthermore, because traditional maps contain extremely limited information dimensions, typically limited to location and geometric information, they struggle to effectively integrate and express multi-dimensional attributes such as vegetation health status, disease identification confidence, problem occurrence time, and historical trends, thus limiting their ability to serve as a comprehensive decision support data source.

[0004] Due to the fixed and uniform data format and content, data generated by traditional maps is difficult for other specialized maintenance systems (such as intelligent irrigation systems, precision fertilization systems, and automated pesticide application equipment) to directly access and parse. The lack of a unified and understandable data exchange medium between these systems hinders the formation of an integrated automated maintenance loop encompassing perception, analysis, decision-making, and execution. Furthermore, traditional mapping... Figure 1 Once established, it typically remains fixed for a considerable period unless there are significant changes in the environmental structure. It cannot dynamically and promptly reflect information about the rapidly changing health status of the lawn, and therefore cannot support predictive maintenance and adaptive maintenance strategy adjustments based on real-time conditions.

[0005] In summary, the core deficiency of existing map models in serving modern, refined agricultural and forestry maintenance lies in their inability to provide a geographic model that can dynamically, multidimensionally, and structurally express the semantic information of lawn health. This deficiency hinders the digitalization and automation of maintenance management from "perception" to "execution." Summary of the Invention

[0006] This invention provides a method, apparatus, device, medium, and management system for constructing a semantic map of lawn health, in order to overcome the deficiencies of the prior art.

[0007] This invention provides a method for constructing a semantic map of lawn health, comprising: For the target lawn area, a multi-layer raster data stack is created, wherein the multi-layer raster data stack divides the target lawn area into multiple regular raster units, each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack, and the multi-layer raster data stack includes a geographic layer and a semantic layer, wherein the semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer; Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; Obtain the set of raster cells corresponding to the current region location information from the multi-layer raster data stack; Based on the lawn health status recognition results, the layer information of each grid cell in the grid cell set is updated according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

[0008] According to the present invention, a method for constructing a semantic map of lawn health is provided, wherein the geographic layer includes a coordinate layer and an elevation layer.

[0009] According to the method for constructing a lawn health semantic map provided by the present invention, the multi-layer raster data stack further includes an extended information layer, which includes a vegetation index (NDVI) layer and a texture layer.

[0010] According to the present invention, a method for constructing a lawn health semantic map includes obtaining the lawn health status identification result of the test area in the target lawn region, which includes: Obtain raw sensing data of the area to be tested, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; Based on the original sensing data, the lawn health status recognition result of the test area is obtained by using a preset lawn health status recognition model.

[0011] According to a method for constructing a semantic map of lawn health provided by the present invention, before acquiring the original perception data of the area to be tested, the method further includes: Construct the preset lawn health status recognition model; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0012] According to the present invention, a method for constructing a lawn health semantic map includes a multi-branch neural network architecture comprising: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0013] According to the present invention, a method for constructing a lawn health semantic map includes a progressive training strategy comprising: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

[0014] According to the method for constructing a semantic map of lawn health provided by the present invention, the adversarial modality dropout training method refers to randomly dropping any number of training data of any number of modalities during the training process of the lawn health status recognition model, so as to simulate the situation of data missing and / or lost.

[0015] According to the present invention, a method for constructing a lawn health semantic map includes the following preset rules: If the original lawn health status is healthy or unknown, or if the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is greater than or equal to the first preset value, the original lawn health status identification result is updated with the current lawn health status identification result. If the current lawn health status is the same as the original lawn health status, update the timestamp; If the current lawn health status is different from the original lawn health status, and the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is less than a second preset value, a conflict negotiation mechanism is triggered. The conflict negotiation mechanism includes any one of the following or any combination thereof: retaining the lawn health status label with higher confidence, marking the lawn health status label as pending confirmation, or making a decision based on the logical priority of the problem type.

[0016] A method for constructing a lawn health semantic map according to the present invention further includes: The updated lawn health semantic map is stored in the time-series map database.

[0017] A method for constructing a lawn health semantic map according to the present invention further includes: Output the updated semantic map of lawn health; Specifically, the output methods include any one of the following or any combination thereof: Visualize the updated semantic map of lawn health; Based on the updated lawn health semantic map, a report containing problem statistics and maintenance recommendations is generated. Push change information or provide data access services to third parties through application programming interfaces (APIs).

[0018] The present invention also provides a lawn health semantic map construction device, comprising: An initialization module is used to create a multi-layer raster data stack for a target lawn area. The multi-layer raster data stack divides the target lawn area into multiple regular raster units. Each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack. The multi-layer raster data stack includes a geographic layer and a semantic layer. The semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer. The identification module is used to obtain the identification result of the lawn health status of the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; The extraction module is used to obtain a set of raster cells corresponding to the current region location information from the multi-layer raster data stack; The update module is used to update the layer information of each grid cell in the grid cell set according to preset rules based on the lawn health status identification results, so as to obtain an updated multi-layer grid data stack as a lawn health semantic map of the target lawn area.

[0019] The present invention also provides an autonomous mobile device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the lawn health semantic map construction method as described in any of the preceding claims.

[0020] The present invention also provides an autonomous mobile device, including a processor and a memory storing a computer program, wherein the processor executes the computer program to implement any of the above-described methods for constructing a lawn health semantic map.

[0021] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described methods for constructing a lawn health semantic map.

[0022] The present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute any of the above-described methods for constructing a lawn health semantic map.

[0023] The present invention also provides a lawn health status management system, including the autonomous mobile device as described above, and a cloud server and / or a user terminal. The cloud server or user terminal is used to receive and display the lawn health status identification results and / or the real-time updated lawn health semantic map sent by the autonomous mobile device.

[0024] The present invention provides a method, apparatus, device, medium, and management system for constructing a lawn health semantic map, which can achieve a fundamental transformation from static navigation maps to dynamic health semantic models, and can bring at least the following beneficial effects: This invention constructs a multi-layered raster data stack, upgrading traditional maps from navigation tools that only contain "accessible / inaccessible" information to lawn health semantic maps that integrate multi-dimensional semantic information such as "what problem, where, confidence level, and when it occurred." Through a standardized, spatially precisely aligned multi-layered structure, the lawn health semantic map achieves structured storage and management of health information, forming a unified data hub connecting perception, decision-making, and execution, thus completely solving the problem of data and application disconnect.

[0025] The machine-readable semantic information in the map, with spatiotemporal labels and confidence levels, can be directly accessed by third-party maintenance systems such as irrigation, fertilization, and pesticide application via API interfaces. This drives these devices to perform precise, targeted, and quantitative operations, significantly saving resources and improving efficiency, which is key to achieving unmanned maintenance. Through conflict resolution and dynamic update mechanisms, the lawn health semantic map can continuously integrate the latest recognition results, maintaining the timeliness of information and providing real-time, reliable data support for adaptive maintenance strategy adjustments, overcoming the shortcomings of traditional static maps.

[0026] The lawn health semantic map utilizes a structured time-series data stack to fully record the historical evolution of lawn health status, providing a solid data foundation for advanced big data analysis such as trend prediction (e.g., disease spread analysis), maintenance effect evaluation, and root cause analysis of problems, empowering scientific decision-making from "experience-driven" to "data-driven".

[0027] The lawn health semantic map, through its layered data stack structure, allows for the flexible integration of new sensing dimensions (such as soil moisture and temperature sensor data) or attribute information as needed in the future, giving it good foresight and continuous evolution capabilities, and protecting long-term investment value.

[0028] The multimodal fusion recognition model employed in this invention integrates appearance, geometry, spectral, and physical information to ensure high accuracy and reliability of the data source input to the semantic map, effectively diagnosing latent problems. Through strategies such as adversarial modality dropout training, the multimodal fusion recognition model maintains stable and reliable recognition performance even in complex real-world environments and when some sensor data is missing, ensuring the continuity and quality of the semantic map data source. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0030] Figure 1 This is a flowchart illustrating a method for constructing a semantic map of lawn health provided by the present invention.

[0031] Figure 2 This is a schematic diagram of the structure of a lawn health semantic map construction device provided by the present invention.

[0032] Figure 3 This is a schematic diagram of the structure of the autonomous mobile device provided by the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, embodiments of this invention, and should not be construed as limiting the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention. In the description of this invention, it should be understood that the terminology used is for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0034] Figure 1 This is a flowchart illustrating a method for constructing a lawn health semantic map according to the present invention. The executing entity of this method can be any applicable terminal-side device or network-side device, such as an automated mobile lawnmower, a cloud server, or a user terminal.

[0035] See Figure 1 The present invention provides a method for constructing a lawn health semantic map, which may include: S110. Create a multi-layer raster data stack for the target lawn area.

[0036] The multi-layer raster data stack divides the target lawn area into multiple regular raster units. Each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack. The multi-layer raster data stack includes a geographic layer, a semantic layer, and an extended information layer. The geographic layer includes a coordinate layer (used to store the latitude and longitude or planar coordinates of the center point of each raster unit) and an elevation layer (used to store the ground elevation value of each raster unit). The semantic layer includes a lawn health status label layer (the initial value of the label is "unknown" or "healthy", and it is updated to a specific problem, such as "groundwater accumulation", "brown spot disease", etc.), a confidence layer (initial value is 0), and a timestamp layer (used to record the time when the label of each raster unit was last updated or confirmed). The extended information layer includes a vegetation index layer (used to store the vegetation index of each raster unit) and a texture layer (used to store the texture feature value of each raster unit).

[0037] S120. Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the current identification result confidence level, the current area location information, and the current timestamp.

[0038] In one embodiment, S120 may include: Obtain raw sensing data of the area to be tested, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; Based on the original sensing data, the lawn health status recognition result of the test area is obtained by using a preset lawn health status recognition model.

[0039] Specifically, the preset lawn health status recognition model can be constructed first; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0040] The multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0041] The progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

[0042] The adversarial modality dropout training method refers to randomly dropping any number of training data for any number of modalities during the training process of the lawn health status recognition model, in order to simulate the situation of data missing and / or lost.

[0043] S130. Obtain the set of grid cells corresponding to the current region location information from the multi-layer grid data stack.

[0044] S140. Based on the lawn health status identification results, update the layer information of each grid cell in the grid cell set according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

[0045] In one embodiment, the preset rules include: When the original lawn health status is healthy or unknown, or when the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is greater than or equal to the first preset value, the original lawn health status identification result is updated with the current lawn health status identification result. Update the timestamp when the current lawn health status is the same as the original lawn health status; When the current lawn health status is different from the original lawn health status, and the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is less than a second preset value, a conflict negotiation mechanism is triggered. The conflict negotiation mechanism includes any one of the following or any combination thereof: retaining the lawn health status label with higher confidence, marking the lawn health status label as pending confirmation, or making a decision based on the logical priority of the problem type.

[0046] S150. Store the updated lawn health semantic map and the corresponding version information into the time-series map database.

[0047] S160. Output the updated lawn health semantic map; Specifically, the output methods include any one of the following or any combination thereof: The updated lawn health semantic map is visualized, for example, by rendering the data stack into a color map that users can understand (e.g., red represents disease and blue represents waterlogging), and applied to an app or webpage. Based on the updated lawn health semantic map, a report containing problem statistics and maintenance recommendations is generated. Push change information or provide data access services to third parties (such as automatic irrigation systems, fertilizer applicators, etc.) through application programming interfaces.

[0048] Compared with existing technologies, this invention, by constructing a dynamic, multi-dimensional, and structured semantic map of lawn health, can bring the following significant benefits: This invention upgrades traditional maps from navigation tools containing only geometric and location information into a "speaking" semantic map that organically integrates multiple dimensions such as lawn health status ("what's wrong"), precise location ("where is it"), diagnostic confidence level, and timestamps. This is equivalent to creating a dynamic "health record card" for the lawn, realizing the digitization and visualization of maintenance management status, and greatly improving the intuitiveness and comprehensibility of the information.

[0049] The multi-layer raster data stack defined in this invention serves as a standardized, multi-layered core data structure and a standardized format for storing and exchanging lawn health data. It acts as a "digital foundation" connecting front-end perception, intermediate analysis, and back-end execution, effectively solving the problem of disconnect between traditional map data and application systems, and providing unified and efficient data support for building an automated maintenance closed loop.

[0050] Because the information in the lawn health semantic map is machine-readable structured data, the health semantic information in it can be directly called by third-party maintenance systems such as irrigation, fertilization, and pesticide application through API interfaces, thereby driving these devices to perform precise operations at fixed points and in fixed quantities, significantly improving operational efficiency and reducing resource consumption. It is a key technological support for realizing unmanned maintenance and smart gardens.

[0051] The dynamic update mechanism implemented in this invention can easily integrate new identification results, enabling online, incremental updates of the lawn health semantic map and ensuring the timeliness of information. Simultaneously, the structured time-series data stack provides the possibility for in-depth big data analysis, such as trend prediction (e.g., disease spread), maintenance effectiveness evaluation, and causal analysis, empowering users to conduct forward-looking management and scientific decision-making.

[0052] The layered data structure used in the lawn health semantic map allows for the easy integration of new sensing dimensions (such as soil moisture and temperature) and attribute information in the future, based on technological development and changes in needs, without changing the core architecture. This gives the entire system good foresight and sustainable evolution capabilities.

[0053] The lawn health semantic map construction device provided by the present invention is described below. The lawn health semantic map construction device described below and the lawn health semantic map construction method described above can be referred to in correspondence.

[0054] See Figure 2 The present invention provides a lawn health semantic map construction device, which may include: An initialization module is used to create a multi-layer raster data stack for a target lawn area. The multi-layer raster data stack divides the target lawn area into multiple regular raster units. Each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack. The multi-layer raster data stack includes a geographic layer and a semantic layer. The semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer. The identification module is used to obtain the identification result of the lawn health status of the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; The extraction module is used to obtain a set of raster cells corresponding to the current region location information from the multi-layer raster data stack; The update module is used to update the layer information of each grid cell in the grid cell set according to preset rules based on the lawn health status identification results, so as to obtain an updated multi-layer grid data stack as a lawn health semantic map of the target lawn area.

[0055] In one embodiment, the geographic layer includes a coordinate layer and an elevation layer.

[0056] In one embodiment, the multi-layer raster data stack further includes an extended information layer, which includes a vegetation index layer and a texture layer.

[0057] In one embodiment, the identification module includes: The test data acquisition module is used to acquire the raw sensing data of the test area, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; The status recognition module is used to obtain the lawn health status recognition result of the test area based on the original sensing data and through a preset lawn health status recognition model.

[0058] In one embodiment, the identification module includes: The model building module is used to build the preset lawn health status recognition model; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

[0059] In one embodiment, the multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

[0060] In one embodiment, the progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

[0061] In one embodiment, the adversarial modality dropout training method refers to randomly dropping any number of training data for any number of modalities during the training process of the lawn health status recognition model, in order to simulate the situation of data missing and / or lost.

[0062] In one embodiment, the preset rules include: When the original lawn health status is healthy or unknown, or when the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is greater than or equal to the first preset value, the original lawn health status identification result is updated with the current lawn health status identification result. Update the timestamp when the current lawn health status is the same as the original lawn health status; When the current lawn health status is different from the original lawn health status, and the difference between the confidence level of the current lawn health status identification result and the confidence level of the original lawn health status identification result is less than a second preset value, a conflict negotiation mechanism is triggered. The conflict negotiation mechanism includes any one of the following or any combination thereof: retaining the lawn health status label with higher confidence, marking the lawn health status label as pending confirmation, or making a decision based on the logical priority of the problem type.

[0063] The present invention provides an autonomous mobile device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the lawn health semantic map construction method as described in any of the above claims.

[0064] This invention provides a lawn health status management system, comprising an autonomous mobile device as described above, a cloud server, and / or a user terminal. The autonomous mobile device is used to acquire the lawn health status identification results. The cloud server or user terminal is used to receive and display the lawn health status identification results and / or real-time updated lawn health semantic maps sent by the autonomous mobile device.

[0065] The following examples illustrate the lawn health semantic map construction method provided by the present invention.

[0066] Example 1: Initial health survey and visualization of a family lawn Process: The user purchases a new smart lawnmower equipped with the features of this invention. After the first full lawnmower, the lawnmower identifies 3 "bald spots," 1 area of ​​"waterlogged area," and several "weed" areas. These results are uploaded, and the system executes steps S110-S160 to generate the user's first semantic map of lawn health. The user clearly sees a color map on their mobile app: bald spots are yellowish-brown, waterlogged areas are dark blue, and weed areas are light green. The map automatically displays: "3 bald spots found, total area 0.5 square meters; 1 area of ​​waterlogged area found..."

[0067] Example 2: Precision maintenance work order for golf course greens Process: The golf course maintenance center's system received an update to the turf health semantic map, showing a 0.8 square meter "brown spot" area on the 18th green with a 92% confidence level. The report generation module immediately created a precise "application work order," which included the geographic boundary file of the area. This file was imported into a precision spraying vehicle or spraying drone, which automatically navigated to the designated location based on the polygon coordinates in the file and sprayed only the 0.8 square meter area, significantly saving on pesticides.

[0068] Example 3: Closed-loop linkage with automatic irrigation system Process: The lawn health semantic map shows that a certain area is marked as "drought" with a confidence level of 85%. This information is pushed to the intelligent irrigation system in real time. After querying the soil type and plant water requirement model of the area, the irrigation system automatically schedules the sprinklers at night to supplement irrigation only in the "drought" area, while keeping other areas normal, thus achieving on-demand water supply.

[0069] Example 4: Dynamic monitoring of disease spread in park lawns Process: The system updates the entire lawn's health semantic map weekly and stores it in a time-series database. Afterward, the maintenance manager can view the dynamic changes of the "Brown Spot Disease" tag layer using the system's timeline slider. The process of the disease gradually spreading outward from a small spot can be clearly seen. Based on this time-series data stack, the system automatically calculates and generates an early warning: "The brown spot disease area is spreading northeast at a rate of 15% per week, and is expected to affect the ornamental area in two weeks," providing a basis for decision-making regarding early isolation and prevention.

[0070] Example 5: Digital Twin Verification of Maintenance Effect Process: A lawn health semantic map marks a "nutrient-deficient" area. The maintenance team fertilizes this area. Two weeks later, the lawnmower returns and finds the area has returned to "healthy." When the system updates the map, it records the transition from "nutrient-deficient" to "healthy." This "action-result" pair is timestamped, and along with the change in NDVI value at that time, it is recorded in the database, forming a digital accumulation of maintenance knowledge for future optimization of fertilization strategies.

[0071] Figure 3 An example is a schematic diagram of the physical structure of an autonomous mobile device, such as... Figure 3 As shown, the autonomous mobile device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, communications interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions stored in the memory 830 to execute the following steps: For the target lawn area, a multi-layer raster data stack is created, wherein the multi-layer raster data stack divides the target lawn area into multiple regular raster units, each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack, and the multi-layer raster data stack includes a geographic layer and a semantic layer, wherein the semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer; Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; Obtain the set of raster cells corresponding to the current region location information from the multi-layer raster data stack; Based on the lawn health status recognition results, the layer information of each grid cell in the grid cell set is updated according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

[0072] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0073] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being able to be stored on a non-transitory computer-readable storage medium, and the computer program being executed by a processor, enabling the computer to perform the following steps: For the target lawn area, a multi-layer raster data stack is created, wherein the multi-layer raster data stack divides the target lawn area into multiple regular raster units, each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack, and the multi-layer raster data stack includes a geographic layer and a semantic layer, wherein the semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer; Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; Obtain the set of raster cells corresponding to the current region location information from the multi-layer raster data stack; Based on the lawn health status recognition results, the layer information of each grid cell in the grid cell set is updated according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

[0074] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps: For the target lawn area, a multi-layer raster data stack is created, wherein the multi-layer raster data stack divides the target lawn area into multiple regular raster units, each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack, and the multi-layer raster data stack includes a geographic layer and a semantic layer, wherein the semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer; Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; Obtain the set of raster cells corresponding to the current region location information from the multi-layer raster data stack; Based on the lawn health status recognition results, the layer information of each grid cell in the grid cell set is updated according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

[0075] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0076] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a semantic map of lawn health, characterized in that, include: For the target lawn area, a multi-layer raster data stack is created, wherein the multi-layer raster data stack divides the target lawn area into multiple regular raster units, each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack, and the multi-layer raster data stack includes a geographic layer and a semantic layer, wherein the semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer; Obtain the lawn health status identification result in the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; Obtain the set of raster cells corresponding to the current region location information from the multi-layer raster data stack; Based on the lawn health status recognition results, the layer information of each grid cell in the grid cell set is updated according to preset rules to obtain an updated multi-layer grid data stack, which serves as a lawn health semantic map of the target lawn area.

2. The method for constructing a lawn health semantic map according to claim 1, characterized in that, The geographic layer includes a coordinate layer and an elevation layer; and / or, the multi-layer raster data stack further includes an extended information layer, which includes a vegetation index layer and a texture layer.

3. The method for constructing a lawn health semantic map according to claim 1, characterized in that, The step of obtaining the lawn health status identification result of the test area in the target lawn area includes: Obtain raw sensing data of the area to be tested, wherein the raw sensing data includes at least one of RGB image blocks, elevation blocks, vegetation index blocks, and real-time vibration data; Based on the original sensing data, the lawn health status recognition result of the test area is obtained by using a preset lawn health status recognition model.

4. The method for constructing a lawn health semantic map according to claim 3, characterized in that, Before acquiring the raw sensory data of the region to be tested, the method further includes: Construct the preset lawn health status recognition model; Specifically, historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and health status labels of the lawn area are obtained. The health status labels include coarse-grained health status labels and problem type labels. Based on the historical RGB image patches, historical elevation patches, historical vegetation index patches, historical vibration data, and the health status labels, a multi-branch neural network architecture is used, employing a progressive training strategy and an adversarial mode dropout training method to train the model to learn the feature changes of the historical RGB image patches, historical elevation patches, historical vegetation index patches, and historical vibration data under different health states, thereby obtaining the lawn health status recognition model.

5. The method for constructing a lawn health semantic map according to claim 4, characterized in that, The multi-branch neural network architecture includes: The visual branch is used to extract visual feature maps of different levels from the historical RGB image blocks and perform visual feature fusion to obtain multi-scale visual feature vectors. The geometric branch is used to extract geometric feature maps of different levels from the historical elevation map tiles and perform geometric feature fusion to obtain multi-scale geometric feature vectors. The spectral branch is used to extract spectral feature maps of different levels from the historical vegetation index map and perform spectral feature fusion to obtain a multi-scale spectral feature vector. The physical branch is used to extract multiple vibration features from the historical vibration data and fuse the vibration features to obtain a physical feature vector. The adaptive branch weight learning module is used to predict the confidence score of each branch on the current sample, calculate the adaptive weight of each branch, and process the output vector of each branch using the adaptive weight of each branch to obtain the weighted vector of each branch. Cross-modal attention fusion network is used to enhance and fuse the weighted vectors of each branch using a cross-modal attention mechanism to obtain a multimodal feature fusion vector; A multi-granularity classifier is used to output the lawn health status recognition result based on the fusion vector of multimodal features.

6. The method for constructing a lawn health semantic map according to claim 4, characterized in that, The progressive training strategy includes: Single-modal pre-training phase: Pre-training of the corresponding branch is performed using training data from only one of the four modalities: visual, geometric, spectral, and physical. Bimodal fusion training phase: Bimodal fusion training is performed using training data from two of the four modalities: visual, geometric, spectral, and physical. Full-modal fusion training phase: Combining training data from four modalities—visual, geometric, spectral, and physical—adaptive branch weight learning module and cross-modal attention fusion network are enabled to conduct full-modal fusion training; Multi-granularity classification training stage: Combining training data from four modalities—visual, geometric, spectral, and physical—and utilizing a multi-task loss function, a main classifier, a fine-grained classifier, and a confidence output layer are trained. The main classifier outputs whether the lawn area to be treated is healthy, the fine-grained classifier outputs the types of problems existing in the lawn area to be treated, and the confidence output layer outputs the model's confidence in the prediction results.

7. The method for constructing a lawn health semantic map according to any one of claims 1-6, characterized in that, The preset rules include: If the original lawn health status in the grid cell corresponding to the current area location information is healthy or unknown, or if the difference between the confidence level of the current identification result and the confidence level of the original identification result is greater than or equal to the first preset value, the original lawn health status identification result is updated with the lawn health status identification result. If the current lawn health status is the same as the original lawn health status, then only the timestamp is updated; If the current lawn health status is different from the original lawn health status, and the difference between the confidence level of the current identification result and the confidence level of the original identification result is less than a second preset value, a conflict negotiation mechanism is triggered. The conflict negotiation mechanism includes any one of the following or any combination thereof: retaining the lawn health status label with higher confidence, marking the lawn health status label as pending confirmation, or making a decision based on the logical priority of the problem type.

8. A lawn health semantic map construction device, characterized in that, include: An initialization module is used to create a multi-layer raster data stack for a target lawn area. The multi-layer raster data stack divides the target lawn area into multiple regular raster units. Each raster unit occupies the same spatial position in all layers of the multi-layer raster data stack. The multi-layer raster data stack includes a geographic layer and a semantic layer. The semantic layer includes a lawn health status label layer, a confidence layer, and a timestamp layer. The identification module is used to obtain the identification result of the lawn health status of the target lawn area, wherein the lawn health status identification result includes the current lawn health status, the confidence level of the current identification result, the current area location information, and the current timestamp; The extraction module is used to obtain a set of raster cells corresponding to the current region location information from the multi-layer raster data stack; The update module is used to update the layer information of each grid cell in the grid cell set according to preset rules based on the lawn health status identification results, so as to obtain an updated multi-layer grid data stack as a lawn health semantic map of the target lawn area.

9. An autonomous mobile device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the lawn health semantic map construction method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the lawn health semantic map construction method as described in any one of claims 1 to 7.

11. A computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, characterized in that, When the computer program is executed by the processor, the computer is able to execute the lawn health semantic map construction method according to any one of claims 1 to 7.

12. A lawn health status management system, characterized in that, Includes the autonomous mobile device as described in claim 9, as well as a cloud server and / or a user terminal, wherein the cloud server or user terminal is used to receive and display the lawn health status identification results and / or the real-time updated lawn health semantic map sent by the autonomous mobile device.

Images