A cloud-edge collaborative control method and system for a mowing robot

Abstract

This application relates to the field of intelligent lawnmower robot control technology, and provides a cloud-edge collaborative control method for lawnmower robots. The method includes: a first control terminal that monitors in real time whether the lawnmower robot experiences a target anomaly; when an anomaly is detected, a first control command is generated based on the anomaly type and sent to the local control terminal to control the lawnmower robot to perform a first escape operation; if the escape operation fails based on real-time physical feedback, a decision is requested from a second control terminal; the second control terminal generates decision information based on the output of a cloud-based large-scale model and sends it to the first control terminal; the first control terminal parses the decision information and generates a corresponding second control command to control the lawnmower robot to perform a second escape operation. Based on this, it can efficiently handle operational anomalies of lawnmower robots, reduce reliance on local control by leveraging cloud-based decision-making, and improve operational reliability and efficiency.

Description

Technical Field

[0001] This application relates to the field of intelligent lawnmower control technology, and in particular to a cloud-edge collaborative control method and system for lawnmowers. Background Technology

[0002] With advancements in sensors and artificial intelligence algorithms, modern lawnmowers possess advanced functions such as environmental perception, autonomous navigation, and intelligent obstacle avoidance, enabling them to complete lawnmower tasks more efficiently and accurately, and providing users with a better experience.

[0003] Currently, most lawnmower robots rely entirely on a local control system for decision-making. This local control system is integrated within the device and receives environmental information from various sensors (such as ultrasonic, infrared, and collision sensors). It then processes and makes decisions locally based on pre-defined algorithms to control the operation of the lawnmower robot, including adjusting mowing speed, changing direction, and performing obstacle avoidance maneuvers.

[0004] However, lawnmowers that rely entirely on local control for decision-making have significant limitations when making unusual decisions in complex real-world scenarios.

[0005] On the one hand, due to cost constraints, the hardware resources (such as processor performance and memory capacity) of the local control unit of the lawnmower robot are limited. This results in insufficient capacity to process complex environmental information and perform advanced algorithm calculations. For example, when the lawn has many irregular obstacles or significant terrain undulations, the local control unit struggles to quickly and accurately analyze the information and make optimal decisions, potentially adopting a conservative obstacle avoidance strategy. This can lead to an unreasonable mowing path, resulting in repeated mowing or missed areas, thus reducing mowing efficiency and quality.

[0006] On the other hand, the algorithms running on the local control terminal are usually pre-installed in the device, and updates and optimizations require manual operation by the user, which is cumbersome. Moreover, due to local hardware limitations, advanced and complex algorithms are difficult to run. When faced with new and abnormal lawn scenarios, the decision-making algorithm cannot be adjusted and optimized in time to adapt to environmental changes. For example, if temporary decorations or rare plants are suddenly erected on the lawn, the local control terminal may not be able to identify and avoid the obstacles appropriately, damaging the obstacles or getting itself into trouble.

[0007] In summary, existing lawnmower robots that rely entirely on local control decisions have many shortcomings in complex scenarios and abnormal situations, making it difficult to meet users' needs for efficient, intelligent, and precise lawnmowering. There is an urgent need to explore new control decision-making methods to break through existing technological bottlenecks. Summary of the Invention

[0008] To address the aforementioned technical problems, this application provides a cloud-edge collaborative control method for a lawnmower robot. The method is applied to a cloud-edge collaborative control system for the lawnmower robot, which includes a first control terminal, a second control terminal, and a local control terminal. The local control terminal runs on the lawnmower robot, and the second control terminal runs on a cloud service. The method includes: during the lawnmower robot's task execution, the first control terminal monitors in real time whether the lawnmower robot experiences a target anomaly; when an anomaly is detected, a first control command is generated based on the anomaly type and sent to the local control terminal to control the lawnmower robot to perform a first escape operation, and based on real-time... Physical feedback determines whether the escape was successful. If the escape was unsuccessful, a structured data sequence is constructed from the local map and relevant historical data feedback according to preset rules and sent to the second control terminal to request a decision from the second control terminal. The second control terminal receives the structured data sequence, constructs a first prompt word based on the structured data sequence, inputs the first prompt word into the cloud-based big data model, generates decision information based on the output of the cloud-based big data model, and sends it to the first control terminal. The first control terminal parses the decision information, generates a corresponding second control command, and sends it to the local control terminal to control the lawnmower robot to perform a second escape operation.

[0009] In one implementation, the method further includes: during the lawnmower robot's task execution, the first control terminal monitors the update status of the local map in real time; when a target change is detected in the local map, a map data summary is generated based on the target change and sent to the second control terminal; the second control terminal receives the map data summary, constructs a second prompt word based on the map data summary, inputs the second prompt word into the cloud-based big data model, generates a task execution strategy based on the output of the cloud-based big data model, and sends it to the first control terminal; the first control terminal extracts the operation parameters from the task execution strategy and sets the operation parameters for the lawnmower robot based on the operation parameters.

[0010] In one implementation, the method for generating a first control command based on an exception type includes: confirming a local escape strategy based on the exception type; extracting a global escape strategy from the task execution strategy; adjusting the local escape strategy to adapt to the global escape strategy to obtain a target escape strategy; and generating the first control command based on the target escape strategy.

[0011] In one implementation, the local map is a multi-dimensional hierarchical semantic map.

[0012] In one implementation, the method for generating the multi-dimensional hierarchical semantic map includes: constructing a static map layer; determining real-time coordinates in the static map layer during movement, and collecting map layer data according to a preset hierarchical architecture, wherein the map layer data includes semantic point cloud data; and mapping the semantic point cloud data to the semantic map layer based on the real-time coordinates.

[0013] In one implementation, the preset layered architecture further includes one or more of an RGB map layer, a slope map layer, and an elevation map layer. Accordingly, the map layered data includes one or more of RGB data, slope data corresponding to the real-time coordinates, and elevation data. The method further includes mapping the RGB data, the slope data, and the elevation data to the RGB map layer, the slope map layer, and the elevation map layer based on the real-time coordinates.

[0014] In one implementation, the method for acquiring the semantic point cloud data includes: performing semantic recognition on the real-time acquired images based on a preset semantic segmentation model, and establishing a mapping relationship between the recognition results and the point cloud data generated by the binocular camera, thereby obtaining two-dimensional semantic point cloud data.

[0015] In one implementation, the first control terminal sends the structured data sequence to the second control terminal based on the Halow network.

[0016] In one implementation, the method by which the second control terminal constructs a first prompt word based on the structured data sequence includes: adding a statement to the first prompt word that limits the decision information output by the cloud-based large model to within a preset behavioral decision threshold.

[0017] Based on the same inventive concept, this application also provides a cloud-edge collaborative control system for a lawnmower robot. The system includes a first control terminal, a second control terminal, and a local control terminal, wherein the first control terminal and the local control terminal run on the intelligent lawnmower robot, and the second control terminal runs on a cloud service. The system is used to execute the above-described method.

[0018] Based on the above technical solution, through the collaborative mechanism between the edge control terminal and the cloud-based large model, when both local and edge terminals fail to escape obstacles, the powerful computing power of the cloud is used to generate decisions, effectively overcoming hardware resource limitations and algorithm rigidity issues. This approach can efficiently handle abnormal operation of the lawnmower robot, reduce manual intervention, and improve operational reliability and efficiency. Furthermore, by monitoring changes in the local map, timely interaction with the cloud-based large model occurs when the target changes. This allows for timely adjustment of operational parameters and enables the cloud-based large model to obtain the latest physical environment information and provide a global escape strategy based on a global perspective. This strategy can then be used by the first control terminal to adjust its local escape strategy, thereby improving the escape efficiency of the edge control terminal. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the cloud-edge collaborative control system for a lawnmower robot provided in one embodiment of this application.

[0021] Figure 2 This is a flowchart of a cloud-edge collaborative control method for a lawnmower robot provided in an embodiment of this application.

[0022] Figure 3 This is a flowchart of the method for generating multi-dimensional hierarchical semantic maps in this application embodiment.

[0023] Figure 4 This is a flowchart of a cloud-edge collaborative control method for a lawnmower robot, provided as another embodiment of this application.

[0024] Figure 5 This is a flowchart of a method for generating a first control instruction based on an exception type in an embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this invention.

[0026] In the description of this application, it should be noted that the terms "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0027] This application provides a cloud-edge collaborative control method for lawnmower robots, applied to... Figure 1 The cloud-edge collaborative control system for the lawnmower robot shown specifically includes a first control terminal 11, a second control terminal 12, and a local control terminal 13.

[0028] The local control terminal 13 is the main control program for the lawnmower robot, used to realize local control of various functional components. It can perform routine basic operations such as lawn mowing, simple escape from obstacles, and obstacle avoidance based on local algorithms. The functional components include: walking component, lawn mowing component, and sensing component.

[0029] The first control terminal 11 can be deployed and run on the lawnmower robot, or mounted on other equipment within the lawnmower robot's activity range, such as a charging pile, an RTK (Real-Time Kinematic) antenna, etc. The first control terminal 11 is communicatively connected to the local control terminal 13 and the second control terminal 12.

[0030] The second control terminal 12 runs on a cloud service, which can be implemented based on a public cloud, private cloud, or hybrid cloud.

[0031] The cloud service also deploys a large-scale cloud model, which is a machine learning model with a large number of parameters and strong generalization capabilities. This model can process complex input information and perform advanced reasoning and decision generation. The second control terminal 12 can access the large-scale cloud model by calling relevant interfaces.

[0032] In implementation, the first control terminal 11 and the second control terminal 12 communicate via a wireless network. In one application example, the first control terminal 11 can communicate with the user terminal router via the Halow wireless network, and then the user terminal router establishes a communication connection with the second control terminal 12, thereby achieving stable data transmission. In another example, the first control terminal 11 can directly communicate wirelessly with the second control terminal 12 via its local network card. Based on this, in some complex working environments, the first control terminal 11 can select the appropriate communication method in real time according to the communication quality with the second control terminal 12 to achieve stable data transmission.

[0033] Based on the above system, the cloud-edge collaborative control method for lawnmowers provided in this application embodiment can combine the cloud, edge, and terminal hierarchical decision-making of the local control terminal 13, the first control terminal 11, and the second control terminal 12 to achieve operation control of the lawnmower. On the one hand, it can reduce equipment costs, and on the other hand, it can effectively utilize the high-order computing power of the cloud to achieve efficient, intelligent, and accurate decision-making. Please refer to... Figure 2 The method specifically includes the following steps.

[0034] S101, during the lawnmower robot's task, when the first control terminal detects a change in the target on the local map, it generates a map data summary based on the target change and sends it to the second control terminal.

[0035] The tasks include mowing the lawn. The local control terminal uses a local map and local control algorithms to control the various functional components to perform these tasks.

[0036] In one implementation, the local control terminal determines environmental information in real time based on images and sensing data collected by the sensing components. When the environmental information changes, the local map is updated. After the local map update is completed, the updated content is sent to the first control terminal. The first control terminal receives the updated content and determines whether the target has changed based on the updated content, thereby realizing real-time monitoring of the update status of the local map.

[0037] In another implementation, the local control terminal sends images and sensing data collected by the sensing components to the first control terminal in real time. The first control terminal monitors environmental information based on this data and updates the local map when it determines that the environmental information has changed.

[0038] Specifically, the local map refers to map data stored locally on the lawnmower robot. In one example, the local map is a multi-dimensional, hierarchical semantic map, generated and updated by the local control unit or the first control unit. Please refer to [link / reference needed]. Figure 3 The method for generating multi-dimensional hierarchical semantic maps specifically includes the following steps.

[0039] S201, Construct a static map layer.

[0040] In implementation, during the map creation phase, users can control the lawnmower robot to walk in the application scenario, determine the map boundary based on the positioning information, and manually delineate the work area and obstacle area to obtain a two-dimensional static map layer containing semantic annotation information such as the work area boundary and obstacle boundary. This provides a spatial reference framework for subsequent real-time semantic mapping, avoiding the need to rebuild the map each time.

[0041] S202, during the movement, determine the real-time coordinates in the static map layer and collect map layer data according to the preset layer architecture.

[0042] In practice, the location information of the lawnmower robot can be obtained in real time based on RTK technology or VSLAM (Visual Simultaneous Localization and Mapping) technology and converted into precise coordinates in a static map, i.e., real-time coordinates.

[0043] The preset layered architecture is a map layering framework pre-built according to application requirements. In this embodiment, the preset layered architecture includes a semantic map layer, an RGB map layer, a slope map layer, and an elevation map layer. Correspondingly, the map layering data to be collected includes semantic point cloud data, RGB data, slope data, and elevation data.

[0044] Specifically, the semantic map layer is used to record semantic point cloud data. In one implementation, semantic recognition can be performed on real-time acquired images based on a preset semantic segmentation model, and the recognition results can be mapped to the point cloud data generated by the binocular camera to obtain two-dimensional semantic point cloud data. The preset semantic segmentation model can be implemented based on a CNN (Convolutional Neural Network) algorithm and can be used to recognize lawn boundaries, obstacles, and location features. It is understood that the model implementation can be set based on actual application requirements to obtain the desired semantic recognition output.

[0045] The RGB map layer is used to record RGB data. In implementation, RGB data can be obtained by recognizing the colors of the collected environmental images to achieve a bird's-eye view similar to Google Maps, presenting a global perspective of the entire application scenario.

[0046] The slope map layer is used to record slope data. In implementation, the slope data can be determined based on the roll angle and pitch angle information obtained by the aircraft's IMU (Inertial Measurement Unit). It can be used to determine whether there is a risk of rollover or slippage in each specific area. Based on this, acceleration, deceleration or walking posture can be optimized. For example, the ability to maintain a straight line when mowing grass along the uphill or downhill slope is much more reliable than that perpendicular to the slope direction, thereby improving walking efficiency and reducing the probability of slippage, rollover, and uneven walking.

[0047] The height map layer is used to record height data. In implementation, height information of all ground can be extracted based on point cloud data. In application, by querying the height map layer, road edges and cliffs with obvious height differences can be extracted, thereby reducing the problem that pure semantic maps have difficulty distinguishing between road edges, cliffs and roads.

[0048] It is worth noting that, apart from the static map layer, the map layer data corresponding to other map layers can be collected in advance by controlling the lawnmower robot to move and collect basic data in various areas within the range defined by the static map layer. During the lawnmower robot's operation, real-time data is collected, and the basic data is updated based on the real-time data. In this way, the map layers can be updated in real time to adapt to changes in the environment.

[0049] S203 maps map layer data to the corresponding map layer based on real-time coordinates.

[0050] In implementation, after collecting data for each map layer, the data can be mapped to the corresponding map layer based on the real-time coordinates of the collected points. Specifically, this involves mapping semantic point cloud data, RGB data, slope data, and height data to the semantic map layer, RGB map layer, slope map layer, and height map layer based on the real-time coordinates. The real-time coordinates are static map layer coordinates converted from the lawnmower robot's real-time positioning coordinates, i.e., the lawnmower robot's position coordinates within the static map layer.

[0051] In this way, based on static map layering, multiple layered maps are constructed and different layered features are mapped and labeled to obtain a multi-dimensional layered semantic map. This not only enables the recording of multi-dimensional features of map data, but also enables two-dimensional storage of map data, thereby reducing the requirements for local memory.

[0052] During the execution of its tasks, the lawnmower robot collects images and sensor information in real time. The first control unit identifies the real-time collected images and sensor information to generate corresponding real-time map layer data. Then, it compares the real-time map layer data with the locally stored local map layer data to determine whether there have been any changes. If there have been any changes, the local map is updated based on the real-time map layer data, thereby achieving real-time updates of the local map.

[0053] In one implementation, when the first control terminal determines that a target change has occurred on the map, it can generate a map data summary based on the updated local map. This map data summary indicates the current environmental characteristics of the lawnmower robot. The first control terminal can determine whether the current change constitutes a target change based on a preset target change range. In one example, target changes include the addition of obstacles, obstacle removal, and height changes exceeding a threshold. It is understood that the target change range can be set according to the actual application scenario.

[0054] In one example, when an obstacle is identified nearby based on real-time collected semantic point cloud data, but the obstacle is not marked on the local map, it is determined that the environmental information has changed. Then, the corresponding obstacle label is added to the local map to complete the map update and generate a map data summary. The map data summary includes the current position of the lawnmower robot, the position and type of the obstacle, for example, the robot is currently located in the eastern part of the lawn, and there is a flower pot 1 meter in front of it.

[0055] S102, the second control terminal receives the map data summary and constructs a second prompt word based on the map data summary.

[0056] In one implementation, the second control terminal can directly write the received map data summary into a pre-built second prompt word template to obtain the second prompt word.

[0057] In another implementation, the second control terminal can first input the map data summary into the cloud big model to request the cloud big model to parse the map data summary according to preset requirements, and then write the parsing result into the pre-built second prompt word template to obtain the second prompt word.

[0058] The second prompt word template contains preset prompt word statements, which are used to provide more detailed application scenario references and to limit the output format and content range, so that the output results are executable.

[0059] S103, input the second prompt word into the cloud big model, generate the task execution strategy based on the output of the cloud big model and send it to the first control terminal.

[0060] In implementation, the cloud-based big model is an ultra-large-scale AI (Artificial Intelligence) model deployed on a cloud server. The second control terminal can call the cloud-based big model through an interface to obtain the output results and generate task execution strategies based on the output results, thereby achieving intelligent decision-making by leveraging the capabilities of the cloud-based big model.

[0061] Specifically, after receiving the output results returned by the large model in the cloud, the second control terminal parses the output results and extracts the content related to task execution, including but not limited to: mowing speed, global path planning and obstacle avoidance safety coefficient, and then converts the extracted task execution content into the work parameters that the mowing robot can execute in the current state, as the task execution strategy.

[0062] S104, the first control terminal extracts the operation parameters from the task execution strategy and sets the operation parameters for the lawn mowing robot based on the operation parameters.

[0063] In implementation, the first control terminal parses the task execution strategy according to preset rules, extracts the operation parameters, and sends the operation parameters to the local control terminal. The local control terminal then resets the operation parameters to control and adjust each functional component according to the reset operation parameters, thereby continuing the pruning process.

[0064] Specifically, the first control unit is responsible for translating the macro-level task execution strategy generated by the cloud-based model into low-level operational parameters that the lawnmower robot can directly understand and execute. For example, operational parameters may include mowing speed, blade height, path point sequence, obstacle avoidance distance, turning radius, and mowing patterns for specific areas (such as spiral mowing or zigzag mowing). The first control unit parses the received strategy file, identifies and extracts these parameters, and then configures them into the lawnmower robot's motion controller, mowing mechanism, and other hardware modules through internal interfaces or drivers. Alternatively, the task execution strategy may contain a series of high-level instructions, such as "avoid area A" or "prioritize mowing area B." The local planning module within the first control unit further refines and generates specific low-level operational parameters based on these high-level instructions and its own real-time perception data, and then sets them accordingly.

[0065] Based on the above solution, for scenarios where the map changes dynamically during lawn mowing operations, an edge-cloud collaboration mechanism is introduced. By leveraging the powerful computing capabilities of the cloud-based large model, a task execution strategy adapted to the new environment is generated, allowing the lawn mowing robot to always operate according to parameters adapted to the current environment. This solves the problem that the local system cannot generate the optimal strategy to cope with dynamic map changes.

[0066] When the first control terminal detects a change in the real-time map during the lawnmower's task, it sends a map data summary instead of the full map data to the second control terminal. This allows the cloud to accurately grasp the core information of the map change, reduces the amount of data transmission, lowers the bandwidth requirements, reduces the amount of data that the cloud needs to process, and improves the response speed of cloud decision-making.

[0067] After receiving the map data summary, the second control terminal constructs a second prompt word input to the cloud-based big model based on the map data summary. Relying on the core information of map changes, it generates prompts, enabling the cloud-based big model to accurately grasp the changes in the current working environment and output decisions that meet the needs of the actual scenario. This fully leverages the advantages of the cloud-based big model's strong computing power and good generalization, eliminating the need for the lawnmower robot's local hardware to carry complex global replanning algorithms. It balances equipment control costs and decision quality. After generating a task execution strategy adapted to the new map, it sends it to the first control terminal, completing the information interaction between the end and the cloud.

[0068] After receiving the task execution strategy, the first control terminal extracts the operation parameters and completes the operation parameter settings for the lawnmower robot. It converts the macro strategy generated in the cloud into parameter configurations that the local lawnmower robot can directly execute, enabling the cloud-based decisions to be quickly implemented and allowing the lawnmower robot to adapt to the changed working environment in a timely manner, ensuring the efficiency and quality of subsequent lawnmower operations.

[0069] In some embodiments of this application, the lawnmower robot can also effectively respond to abnormal conditions by relying on hierarchical decision-making at the cloud, edge, and terminal levels. Please refer to... Figure 4 Another embodiment of this application provides a cloud-edge collaborative control method for a lawnmower robot, which specifically includes the following steps.

[0070] S301: During the lawnmower's task execution, the first control terminal monitors the lawnmower's operating status in real time to determine whether any target anomalies have occurred.

[0071] During implementation, the first control unit continuously monitors for any target anomalies while the lawnmower is performing its tasks.

[0072] Anomalies in targets can be monitored based on a variety of information sources, including but not limited to local maps, real-time physical feedback, and historical status information.

[0073] In one example, the first control unit can rely on the local map and current location to determine whether the robot has gone out of bounds (out of the work area), encountered obstacles, and the current height and slope of the area. Alternatively, it can rely on real-time physical feedback, such as detecting motor current, wheel speed, or collision sensor data to determine whether the lawnmower robot is stuck, slipping, tilting, or lifting. It can also rely on historical state information, analyzing the similarity between recent operating trajectories and historical anomaly records to predict potential anomalies, or determining the confidence level of the current location based on historical state information. It is worth noting that when determining whether tilting or lifting has occurred, the local map and real-time physical feedback can be combined to determine whether the tilting or lifting is caused by the terrain.

[0074] If a target anomaly is detected, proceed to step S302; if no target anomaly is detected, continue monitoring.

[0075] The types of target anomalies include, but are not limited to, path planning failure, drive slippage, collision, fuselage lifting or tilting, going out of bounds, positioning disbelief, or grass cutting blockage. It is understood that the types of target anomalies can be set according to actual application requirements. For example, they can be set according to the anomaly recovery capability of the local control terminal. Anomalies that the local control terminal cannot recover from in time can be identified as target anomalies, while anomalies that can recover from in time can be identified as ordinary anomalies. This application is not limited to this.

[0076] It is worth noting that common anomalies encountered during task execution are handled by the local control terminal using local algorithms. Based on this, by classifying the difficulty of anomaly resolution and setting corresponding layered resolution algorithms, the capabilities of the local control terminal can be fully utilized, reducing interaction between control terminals and thus lowering system communication pressure.

[0077] S302 generates the first control command based on the exception type and sends it to the local control terminal.

[0078] In implementation, different types of target anomalies correspond to different escape strategies. The first control terminal can match the corresponding local escape strategy from the locally stored escape strategies based on the anomaly type detected by the target anomaly, and generate a first control command based on the local escape strategy and send it to the local control terminal to instruct the lawnmower robot to perform an escape operation. The local escape strategy stores motor control commands that the local control terminal can directly execute. The local control terminal executes these control commands to control each functional component to perform the corresponding escape actions.

[0079] For example, when a robot is detected to be stuck, instructions can be generated to make it attempt to move backward, turn, or vibrate slightly. When a robot is detected to have tipped over, instructions can be generated to make it attempt to adjust its posture.

[0080] In some embodiments of this application, please refer to Figure 5 The method for generating the first control instruction based on the exception type specifically includes the following steps.

[0081] S401, confirm the local escape strategy based on the anomaly type.

[0082] In practice, the first control terminal can match the corresponding local escape strategy from the local system based on the anomaly type corresponding to the detected target anomaly.

[0083] S402, extract the global escape strategy from the task execution strategy, and adjust the local escape strategy based on the global escape strategy to obtain the target escape strategy.

[0084] exist Figure 2 In the embodiment shown, the task execution strategy generated by the second control terminal based on the output of the cloud-based large model may also include a global escape strategy.

[0085] Specifically, during the process of constructing the second prompt word based on the map data summary, the second control terminal can add a limiting statement to the second prompt word that requires the connection to the environmental context during the execution of this task to generate a global escape strategy corresponding to each anomaly type. In this way, the task execution strategy containing the global escape strategy corresponding to each anomaly type can be obtained in the output of the large cloud model.

[0086] Based on this, by leveraging the contextual understanding capabilities of the cloud-based big data model, the cloud-based big data model can gain a global perspective on the changes in the physical environment during the execution of this task. In this way, a global escape strategy adapted to the current task execution can be obtained, and this global escape strategy can be dynamically updated as the dialogue content during this task is updated, thereby continuously adjusting to adapt to the complex and changing external environment.

[0087] The global escape strategies for each type of anomaly can be the same or different. In one example, when the cloud-based big model determines that it is raining based on the dialogue content during the task, the output global escape strategy for the target anomaly of "blocking the lawn" is to extend the waiting time or to wait in place and send a reminder to the user.

[0088] The method for adjusting a local escape strategy based on a global escape strategy involves comparing whether there is a conflict between the two strategies. If a conflict exists, the global escape strategy takes precedence; if no conflict exists, the local escape strategy is adjusted according to the requirements of the global escape strategy. For example, if the global escape strategy is to wait in place, and the local escape strategy is to attempt to reverse and move, then a conflict exists, and the global escape strategy is determined to be the target escape strategy. If the global escape strategy is to extend the waiting time, the local escape strategy is adjusted to wait for 5 minutes before attempting to reverse and move.

[0089] S403 generates the first control command based on the target escape strategy.

[0090] In practice, the first control terminal can determine the corresponding motor control command based on the target escape strategy and generate the corresponding first control command.

[0091] By dynamically optimizing the local escape strategy based on the global escape strategy, the local escape strategy can be adapted to complex and ever-changing application environments, thereby improving the effectiveness of the escape strategy. This not only avoids wasted effort in scenarios unsuitable for autonomous escape, but also increases the success rate of autonomous escape by optimizing the escape actions.

[0092] S303 determines whether a person has successfully escaped a difficult situation based on real-time physical feedback.

[0093] In implementation, after sending the first control command, within a preset escape time, the first control terminal determines whether the escape has been successful through real-time physical feedback to the lawnmower robot. For example, it determines whether the robot has resumed movement by monitoring wheel speed sensor data, or whether the robot has returned to a normal posture by monitoring posture sensor data. If the lawnmower robot escapes the target abnormal state within the preset escape time and its state remains normal, the escape is considered successful; if the lawnmower robot remains in the target abnormal state or repeatedly experiences the target abnormality within the preset escape time, the escape is considered unsuccessful. The preset escape time can be set based on empirical values, and the preset escape time for different abnormality types can be set separately. By continuously monitoring within the preset escape time, the success rate of the escape determination can be guaranteed to a large extent, avoiding judgment errors caused by temporary parameter changes.

[0094] S304. If it is determined that the escape has failed, according to preset rules, the local map and relevant historical data feedback are constructed into a structured data sequence and sent to the second control terminal to request the second control terminal to issue a decision.

[0095] Among them, the relevant historical data includes real-time physical feedback data when the lawnmower robot is in an abnormal state of the target, historical diagnostic logs, etc. In one implementation, the structured data sequence includes image data collected by binocular cameras, intrinsic and extrinsic parameters of various sensors, local map local tiles, real-time physical state vectors (such as coordinates, slope, IMU current, wheel speed current) and historical diagnostic logs, which can provide relevant data that can provide a complete physical environment context.

[0096] Structured data sequences can be constructed by simply packaging all the raw data into a single file, or by stitching together sensor data and map snapshots in chronological order.

[0097] Data can be sent to the second control terminal through various communication methods, such as Wi-Fi, cellular networks, or Bluetooth connections.

[0098] In one example, the first and second control terminals can communicate via the Halow wireless network. Specifically, the first control terminal can establish wireless communication with user routers, charging stations, or RTK antennas through the Halow wireless network, and then establish a communication connection with the second control terminal through these devices. Based on this, the stability of network communication can be guaranteed when the first control terminal is mounted on a lawnmower robot.

[0099] S305, the second control terminal receives the structured data sequence and constructs the first prompt word based on the structured data sequence.

[0100] In implementation, the first prompt word can be constructed by directly concatenating key information from the structured data sequence into a text string according to a preset template, and adding relevant limiting statements to clarify the application scenario and output requirements. This includes adding statements to the first prompt word that limit the decision information output by the cloud-based large model to a preset behavioral decision domain.

[0101] For example, after a failed attempt to escape a skid-induced obstacle, the generated prompt is: "Assuming you are a regulatory decision-making algorithm expert, you need to combine data from the front-facing binocular camera, multi-layer semantic map data, current physical state data, sensor intrinsic and extrinsic parameters, and historical diagnostic logs to make a decision. I will then provide you with the input in sequence. Please answer 'Yes' for each input received. When you receive: Please make a decision and output the following decision in the format below:" { "arm_safe": "YES|NO|LIMITED", "plan": [ { "a": "STOP"}, { "a": "LOCK_BLADE"}, { "a": "ARM_INSPECT", "p": { "t": 2.0}}, { "a": "ARM_GRASP_PULL", "p": { "f": 8.0, "t": 3.0}}, { "a": "ARM_RETRACT"} ], "mark": { "mode": "NO|TEMP|PERM", "type": "WIRE|VINE|ENTANGLE|UNKNOWN", "xy": [0.0, 0.0], "r": 0.0, "ttl": 0 } } Field Description arm_safe: Whether to allow the robotic arm to deploy (LIMITED = Allowed, but with a conservative strategy) plan[]: Action sequence a = action (action enumeration) p = params (optional parameters) It is recommended to keep only the following parameters: f (maximum force in N), t (time in seconds), d (distance in meters), and yaw (angle in degrees). mark: Whether to mark the plot mode: Unmarked / Temporary / Permanent type: Risk type xy: Mark the center (map coordinates) r: radius (meters) TTL: Temporary tag validity period (seconds); permanent can be set to 0. Action enumeration suggestion (as few as possible) Robot side: STOP, BACK_OFF, REPOSITION Security side: LOCK_BLADE Robotic arm side: ARM_INSPECT, ARM_PROBE, ARM_GRASP_PULL, ARM_RETRACT Exit: ALERT_USER Based on this, the output of the large cloud-based model can be limited to content that the program can recognize. It is understandable that different prompt templates can be set for different escape failure scenarios to adapt to different output requirements.

[0102] S306: Input the first prompt word into the cloud-based big model, generate decision information based on the output of the cloud-based big model, and send it to the first control terminal.

[0103] In practice, the second control terminal can determine the output results of the cloud-based large model as decision information and send it to the first control terminal.

[0104] S307, the first control terminal parses the decision information, generates a corresponding second control command, and sends it to the local control terminal to control the lawnmower robot to perform the second escape operation.

[0105] In practice, the first control terminal can parse the decision information in the second control command according to the preset parsing rules to obtain the motor control command that the local control terminal can directly execute, and send it to the local control terminal in the form of the second control command. After receiving the second control command, the local control terminal prioritizes the motor control command in it to control each functional component to perform the corresponding escape action.

[0106] Based on this, by introducing a cloud-edge collaborative control mechanism, the problem of insufficient decision-making ability in complex scenarios and abnormal situations when traditional lawnmowers rely entirely on local control terminals for decision-making is effectively solved. Through the division of labor and cooperation between the first and second control terminals, combined with the rapid local response and the powerful information processing and decision-making capabilities of the cloud-based large model, the lawnmower can more comprehensively monitor anomalies and generate more reasonable and refined decisions for complex escape scenarios, thereby improving the autonomous operation efficiency and task completion quality of the lawnmower in changing environments.

[0107] The above description is merely a preferred embodiment of this application and is not intended to limit this application in any way. Although this application has been disclosed above with reference to preferred embodiments, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A cloud-edge collaborative control method for a lawnmower robot, characterized in that, The method is applied to a cloud-edge collaborative control system for a lawnmower robot. The system includes a first control terminal, a second control terminal, and a local control terminal. The local control terminal runs on the lawnmower robot, and the second control terminal runs on a cloud service. The method includes: During the lawn mowing robot's task execution, the first control terminal monitors in real time whether the lawn mowing robot experiences any target anomalies. When the target abnormality is determined to occur, a first control command is generated according to the abnormality type and sent to the local control terminal to control the lawnmower robot to perform the first escape operation, and to determine whether the escape is successful based on real-time physical feedback; If the escape attempt fails, the local map and relevant historical data feedback are constructed into a structured data sequence according to preset rules and sent to the second control terminal to request the second control terminal to issue a decision. The second control terminal receives the structured data sequence, constructs a first prompt word based on the structured data sequence, inputs the first prompt word into the cloud-based big model, generates decision information based on the output of the cloud-based big model, and sends it to the first control terminal. The first control terminal parses the decision information, generates a second control command, and sends it to the local control terminal to control the lawnmower robot to perform a second escape operation.

2. The method according to claim 1, characterized in that, The method further includes: During the lawnmower's task execution, the first control terminal monitors the update status of the local map in real time. When a target change is detected in the local map, a map data summary is generated based on the target change and sent to the second control terminal. The second control terminal receives the map data summary, constructs a second prompt word based on the map data summary, inputs the second prompt word into the cloud big model, generates a task execution strategy based on the output of the cloud big model, and sends it to the first control terminal. The first control terminal extracts the operation parameters from the task execution strategy and sets the operation parameters for the lawn mowing robot based on the operation parameters.

3. The method according to claim 2, characterized in that, The method for generating the first control command based on the exception type includes: Based on the anomaly type, a local escape strategy is determined, a global escape strategy is extracted from the task execution strategy, and the local escape strategy is adjusted to adapt to the global escape strategy to obtain the target escape strategy. The first control command is generated based on the target escape strategy.

4. The method of claim 2, wherein, The local map is a multi-dimensional, layered semantic map.

5. The method of claim 4, wherein, The method for generating the multi-dimensional hierarchical semantic map includes: Construct a static map layer; During the movement, the real-time coordinates in the static map layer are determined, and map layer data is collected according to a preset layer architecture, wherein the map layer data includes semantic point cloud data; The semantic point cloud data is mapped to the semantic map layer based on the real-time coordinates.

6. The method according to claim 5, characterized in that, The preset layered architecture further includes one or more of an RGB map layer, a slope map layer, and an elevation map layer. Correspondingly, the map layered data includes one or more of RGB data, slope data corresponding to the real-time coordinates, and elevation data. The method further includes mapping the RGB data, the slope data, and the elevation data to the RGB map layer, the slope map layer, and the elevation map layer based on the real-time coordinates.

7. The method according to claim 5, characterized in that, The method for acquiring the semantic point cloud data includes: performing semantic recognition on the real-time acquired images based on a preset semantic segmentation model, and establishing a mapping relationship between the recognition results and the point cloud data generated by the binocular camera, thereby obtaining two-dimensional semantic point cloud data.

8. The method according to claim 1, characterized in that, The first control terminal sends the structured data sequence to the second control terminal based on the Halow network.

9. The method according to claim 1, characterized in that, The method by which the second control terminal constructs the first prompt word based on the structured data sequence includes: Add a statement to the first prompt word that limits the decision information output by the cloud-based big model to a preset behavioral decision domain.

10. A cloud-edge collaborative control system for a lawnmower robot, characterized in that, The system includes a first control terminal, a second control terminal, and a local control terminal, wherein the first control terminal and the local control terminal run on an intelligent lawnmower robot, the second control terminal runs on a cloud service, and the system is used to execute the method of any one of claims 1 to 9.

Images