LLM-based cleaning robot task management method

Abstract

The invention relates to the technical field of cleaning robot task management, and discloses an LLM-based cleaning robot task management method, which comprises the steps of analyzing a user voice instruction in real time through an LLM-driven natural language interaction engine, generating a structured task description, dynamically optimizing task priorities and resource allocation in combination with multi-modal perception data, and carrying out task management on the user voice instruction. The method achieves the autonomous decision-making of environment abnormity, supports the real-time change of tasks and the closed-loop learning of historical data in the cleaning process, and forms a personalized service closed-loop mechanism. According to the method, the man-machine interaction naturalness, the task execution efficiency and the system self-adaptive capacity are remarkably improved, and the technical problems that in the traditional technology, semantic understanding is lacked, environmental perception is insufficient, resource distribution is rigid and personalized services are lacked are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot task management, and in particular to a cleaning robot task management method based on LLM. BACKGROUND

[0002] Traditional cleaning robot task management relies on preset programs or rule engines, and can only handle a fixed instruction set (such as key selection or APP preset options), and cannot parse complex semantics in natural language (such as "first handle kitchen oil stains and then quickly clean the living room"). This rigid interaction mode requires users to adapt to machine logic, resulting in low instruction input efficiency, high misoperation rate, and inability to support multi-intent combination or ambiguous demand expression, which seriously restricts the naturalness and convenience of human-machine collaboration.

[0003] Existing systems lack real-time analysis capability for dynamic environments and cannot integrate multi-source sensor data (such as obstacle data identified by laser radar and stain data identified by vision module) for autonomous decision-making. When the environment changes suddenly (such as the appearance of temporary obstacles or liquid stains), the robot can only execute preset obstacle avoidance logic or interrupt the task, and cannot dynamically adjust the cleaning strategy (such as switching the suction mode or detouring the path), resulting in frequent path conflicts in complex scenarios, insufficient cleaning coverage, and difficulty in meeting the diversity needs of home scenarios.

[0004] Traditional technology uses fixed priority sorting (such as by area size or preset order), without combining real-time state (power, stain urgency, hardware load) for dynamic optimization. When the task duration exceeds the battery life or the hardware is overloaded, the system cannot intelligently generate temporary charging schemes or task rearrangement logic, and can only passively interrupt the operation, causing task fragmentation and repeated machine startup, reducing overall operation efficiency and user experience.

[0005] Existing systems do not support dynamic changes in tasks during cleaning (such as temporarily skipping an area or adjusting the mode), and users need to forcibly stop the robot and reconfigure it, which cannot meet sudden needs. At the same time, the system does not establish a data loop for user behavior, and cannot learn preferences (such as area priority, cleaning intensity habit) through historical instructions, change records, etc., resulting in serious service homogeneity and difficulty in achieving personalized task planning and continuous optimization based on user profiles. SUMMARY

[0006] The present application is a cleaning robot task management method based on LLM to solve the problems in the prior art.

[0007] In order to achieve the above purpose, the present application adopts the following technical scheme: a cleaning robot task management method based on LLM, comprising the following steps: Step S1: The user starts the cleaning robot task management system through instructions and enters the task management interface; Step S2: The user selects a work control through the task management interface; Step S3, the task management system generates a work task according to the user's selection of the work control; Step S4, prioritizing the work tasks and allocating resources, generating and sending a work plan to the cleaning robot; Step S5: Establish a work path, drive the cleaning robot to execute the work plan, and perform cleaning work; Step S6: The cleaning robot obtains status feedback in real time during the cleaning process and dynamically adjusts the task execution process according to the status feedback; Step S7: If the user changes the work task during the cleaning process, the system will immediately change the work plan based on the user's changes and status feedback; Step S8: storing status feedback, user instructions, work tasks, work history and change records in a database.

[0008] Furthermore, in step S1, the following sub-steps are also included: S1-1, the user issues a voice start command to the cleaning robot, or starts the cleaning robot task management system by pressing a physical start button or a virtual start button on a mobile app, wherein the physical start button is located on the cleaning robot body, and the mobile app is installed on the user's mobile device, which is connected to the cleaning robot via a wireless network; S1-2, when the cleaning robot captures the voice start command, it transcribes the voice command into text through automatic speech recognition (ASR) to determine whether it contains the start keyword. If so, it starts the cleaning robot task management system; S1-3, when the cleaning robot task management system is started, the task management interface is displayed. The task management interface provides work controls through a visualization method, distinguishes the functional levels of the work controls, and determines the mapping relationship between the work controls and the robot's work tasks. The work tasks include cleaning areas, cleaning time, and cleaning modes.

[0009] Furthermore, in step S2, the following sub-steps are also included: S2-1, the user sends a voice selection command to the cleaning robot, or selects the working control through the direct touch screen control method of the task management interface or the remote touch screen control method of the mobile terminal APP; S2-2, when the cleaning robot captures the user's voice selection command, it transcribes the voice selection command into text through automatic speech recognition (ASR), and uses the LLM to call the intent recognition model to parse the user's selection intention, and then automatically matches the work control options according to the user's selection intention; S2-3, after the user completes the selection, the task management interface and the mobile APP display the work control selected by the user and its work status.

[0010] Furthermore, in step S3, the following sub-steps are also included: S3-1, the control system converts the user's selection of work controls into structured data and generates a structured task description, including work area coordinates, cleaning parameters, and constraints; S3-2, generating a work task based on the structured task description, historical cleaning records, and a room layout diagram, wherein the historical cleaning records are obtained from a database, and the room layout diagram is generated by user input in the task management interface; S3-3, decompose the work task into multiple subtasks, wherein the subtasks include basic tasks and combined tasks.

[0011] Furthermore, in step S4, the following sub-steps are also included: S4-1, read the work task and verify its data integrity; S4-2, check whether there are conflicts between the subtasks of the work task. If so, adjust the task priority through LLM and generate a priority identifier; S4-3, check the hardware requirements of the subtasks in the work task, calculate the estimated time and energy consumption of each subtask, and generate a temporary charging plan through LLM if the total time exceeds the battery life; S4-4, generates a work plan based on the priority identification and temporary charging plan, and sends it to the cleaning robot through the MQTT protocol.

[0012] Furthermore, in step S5, the following sub-steps are also included: S5-1, the cleaning robot receives the work plan, where the work plan includes a work area, a task execution priority, and a charging point; S5-2, establish a SLAM map model and map the work area to the SLAM map model. If the work task involves multiple work areas, calculate the optimal area traversal order through LLM, use the hybrid A* algorithm to generate an S-shaped coverage path to obtain the work path; S5-3, the cleaning robot performs cleaning work along the working path.

[0013] Furthermore, in step S6, the following sub-steps are also included: S6-1: During the cleaning process, the cleaning robot obtains real-time feedback on its own status and environmental status. The self-status feedback includes hardware status and task completion status. The hardware status includes the battery management system (BMS), motor current, water tank level, and roller brush speed. The task completion status includes task progress and path deviation. The environmental status feedback includes obtaining the coordinates of surrounding obstacles through the lidar, identifying the ground material and stain type through the vision module, and detecting abnormal noise through the sound sensor. S6-2, combining the cleaning robot's state feedback and historical abnormality data to perform state anomaly detection through LLM to identify abnormal states, wherein the state feedback includes self-state feedback and environmental state feedback, and the abnormal states include hardware abnormalities, task abnormalities, and environmental abnormalities; S6-3, LLM generates a dynamic task adjustment strategy based on the abnormal state. If multiple abnormal states exist, multi-objective optimization is called to generate adjustment priorities.

[0014] Furthermore, in step S7, the following sub-steps are also included: S7-1, the user issues a voice command to the cleaning robot to change the work task; S7-2, the cleaning robot captures the user's voice command, transcribes the voice command into text through automatic speech recognition (ASR), and uses the LLM to call the intent recognition model to analyze the user's change intention, which includes adding tasks, reducing tasks, and the number and type of tasks added or reduced; S7-3, LLM regenerates and sends the changed work plan to the cleaning robot based on the user's change intention and status feedback; S7-4, the cleaning robot performs cleaning work according to the modified work plan; S7-5, the task management system sends the modified task plan to the mobile device via the wireless network, displays the modified task plan on the mobile app, and announces the change results through voice feedback TTS.

[0015] Furthermore, in step S8, the following sub-steps are also included: S8-1, establishing a hierarchical database, wherein the hierarchical database includes a time series database, a relational database, and a vector database; S8-2, storing state feedback via a time series database; The association between work tasks and user instructions is established through a relational database and stored in a table format. User instructions include key commands, mobile APP operation commands, and voice commands. The vector database stores work history and change records, including historical cleaning records and change history retained when users change work tasks; S8-3, when storing the user's voice command, the hierarchical database automatically detects and shields personal identity information in the voice command through LLM sensitive word recognition.

[0016] The beneficial effects brought about by the technical solution provided by the present invention include at least: The present invention builds a natural language interaction engine by integrating a large language model (LLM), parses user voice commands in real time and converts them into structured text, and combines automatic speech recognition (ASR) and intent recognition models to accurately map fuzzy requirements into task descriptions with constraints (such as area coordinates, cleaning parameters, physical / time constraints). The semantic understanding ability of LLM significantly improves the naturalness of human-computer interaction, allowing non-professional users to directly control the robot through natural language without learning complex operations, while ensuring the integrity and executableness of task instructions, solving the problems of rigid interaction and high error rate of instruction parsing in traditional systems.

[0017] This invention utilizes LLM to analyze task status (such as power consumption, hardware load, and environmental complexity) and historical data in real time, dynamically adjusting the priorities and resource allocation strategies of multiple subtasks. When task conflicts are detected (such as a high-energy-consuming task and a low-battery task), LLM automatically generates temporary recharging plans or task rescheduling logic to ensure efficient execution of the task chain within resource constraints. Compared to traditional static rules, LLM's real-time decision-making capabilities significantly reduce ineffective energy consumption, avoid task interruptions due to insufficient resources, and significantly improve system robustness.

[0018] The present invention uses LLM to fuse multimodal data such as lidar, vision modules, and sound sensors to identify hardware failures (such as motor overload), environmental risks (such as liquid stains), and task deviations (such as path deviation) in real time, and autonomously generate dynamic adjustment strategies (such as reducing the speed, switching the cleaning mode, or replanning the path), achieving seamless task connection in abnormal scenarios. This allows the robot to autonomously respond to complex environmental changes without human intervention, ensuring the continuity and safety of cleaning operations.

[0019] This invention uses a vector database to store work history and change records, supports LLM semantic retrieval, and dynamically updates task strategies by analyzing user behavior patterns (such as task change frequency and regional priority adjustments), forming a closed-loop learning system of "interaction-execution-feedback-optimization". Compared with traditional systems, this mechanism not only improves the accuracy of task matching, but also ensures data security through privacy protection mechanisms (such as sensitive word filtering), ultimately achieving a synergistic improvement in user satisfaction and system intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 A flowchart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0022] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a cleaning robot task management method based on LLM proposed in accordance with the present invention. In the following description, different references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0023] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0024] The following examples are for illustrative purposes only and are not intended to limit the scope of the present invention.

[0025] The following describes in detail a specific solution of a cleaning robot task management method based on LLM provided by the present invention with reference to the accompanying drawings.

[0026] See also Figure 1 , which shows a method flow chart of a cleaning robot task management method based on LLM provided by an embodiment of the present invention, the method comprising the following steps: Step S1: The user starts the cleaning robot task management system through instructions and enters the task management interface; Wherein, step S1 further includes the following sub-steps: S1-1, the user issues a voice start command to the cleaning robot, or starts the cleaning robot task management system by pressing a physical start button or a virtual start button on a mobile app. The physical start button is located on the cleaning robot body, and the mobile app is installed on the user's mobile device, which is connected to the cleaning robot via a wireless network. S1-2, when the cleaning robot captures the voice start command, it transcribes the voice command into text through automatic speech recognition (ASR) to determine whether it contains the start keyword. If so, it starts the cleaning robot task management system; S1-3, when the cleaning robot task management system is started, a task management interface is displayed, the task management interface provides work controls through a visual method, and the functions of the work controls are distinguished and the mapping relationship between the work controls and robot work tasks is determined, the work tasks including a cleaning area, a cleaning time length and a cleaning mode.

[0027] It should be noted that automatic speech recognition (ASR) is a technology for converting human speech into text in real time, and the core process includes: speech signal preprocessing (noise reduction, framing), acoustic model (extracting MFCC features), language model (predicting word sequence), and decoder (outputting optimal text). Modern ASR systems (such as Whisper, DeepSpeech) are based on deep neural networks, support multiple languages, dialects and noisy environments, and are widely used in intelligent assistants, voice input and real-time subtitles, etc.

[0028] Step S2, the user selects a work control through the task management interface; In step S2, the following sub-steps are further included: S2-1, the user issues a voice selection instruction to the cleaning robot, or selects a work control through a direct touch screen control method, a mobile terminal APP remote touch screen control method; S2-2, when the cleaning robot captures the user voice selection instruction, the voice selection instruction is transcribed into text through automatic speech recognition ASR, and an intent recognition model is called through LLM to analyze the user selection intent, and then the work control option is automatically matched according to the user selection intent; S2-3, after the user completes the selection, the task management interface and the mobile terminal APP display the work control selected by the user and the working state thereof.

[0029] It should be noted that the task management interface realizes human-computer interaction through layered visual work controls (such as area selection, time axis, cleaning mode slider), and the work controls are divided according to function levels, for example: The basic layer includes area / mode selection, and the advanced layer includes path optimization / charging strategy; The LLM analyzes the user's operation on the work control, and establishes a dynamic mapping relationship between the work control parameters and the robot task instructions, so as to ensure that the user's intuitive operation is directly converted into an executable work plan, for example, the work control parameter "kitchen + deep cleaning"→robot task instruction: The large language model (LLM) calls a lightweight intent recognition model through fine-tuning or prompting engineering to quickly analyze the user input into a structured intent, including: Coarse-grained intent: The LLM receives user input (e.g., "Stop cleaning the kitchen"), combines it with the context (current task status, historical preferences), generates a list of candidate intents (e.g., "Skip the kitchen," "Pause the task"), and outputs a confidence score. Fine-grained intent: LLM calls a lightweight intent recognition model (such as a fine-tuned classifier based on BERT) to perform semantic verification and conflict detection on candidate intents. For example: If the robot is cleaning the kitchen, the model will mark "skip the kitchen" as a high-risk action, triggering a second confirmation. If the user says "focus on cleaning the living room", the model will interpret it as: Dynamic feedback optimization: Intent recognition results are fed back to the LLM to update user profiles (e.g., "users often skip the kitchen" → subsequently lowering the kitchen's priority by default), forming a closed-loop learning process.

[0030] This architecture complements the generalization capabilities of LLM and the precise classification of the intent model. LLM is responsible for semantic understanding and contextual reasoning, while the intent model focuses on high-precision classification. The two work together to achieve precise conversion from natural language to machine instructions, enabling efficient parsing of complex instructions and real-time adjustment of robot tasks, while reducing computing latency (intent model response <50ms).

[0031] Step S3, the task management system generates a work task according to the user's selection of the work control; Wherein, in step S3, the following sub-steps are also included: S3-1, the control system converts the user's selection of work controls into structured data and generates a structured task description, including work area coordinates, cleaning parameters, and constraints; S3-2, generating work tasks based on structured task descriptions, historical cleaning records, and room layout diagrams. Historical cleaning records are obtained from the database, and room layout diagrams are generated by user input in the task management interface. S3-3, decompose the work task into multiple subtasks, which include basic tasks and combined tasks.

[0032] It should be noted that the working area coordinates: the control system obtains the precise boundaries of the user's selected area (such as kitchen, bedroom) in real time through the SLAM map and marks the location of obstacles, converting it into a two-dimensional coordinate array. The coordinate data is stored in JSON format, including the area ID, type (hardwood / carpet) and coverage priority, to ensure that the robot's path planning can accurately cover the target area to avoid omissions or repeated cleaning.

[0033] Cleaning parameters: Dynamically generated by the user-selected work controls, including suction level (such as 3000Pa), water volume (such as 0.3L / min), roller brush speed (such as 120rpm) and cleaning mode (such as "deep / fast"). The system detects the area type and automatically optimizes parameters based on the area type (such as increasing suction in carpet areas and reducing water volume in hardwood areas). The rationality of the parameters is verified through LLM (such as disabling high-power mode when the battery is low) to ensure a balance between cleaning effect and energy consumption.

[0034] Constraints: These define the rules limiting the robot's mission execution, including physical constraints (such as avoiding the coordinates of a pet bowl), time constraints (such as silence during lunch breaks), resource constraints (such as interrupting the mission when the battery level drops below 15%), and user preferences (such as avoiding the edge of a carpet). LLM dynamically updates these constraints based on real-time environmental data (such as the pet's location) to generate a machine-readable rule set to ensure safe and compliant mission execution.

[0035] The room layout map is a two-dimensional spatial model that the user manually uploads or edits through the task management interface. It contains key information such as room boundaries, furniture locations, and floor material partitions (such as sofa coordinates and carpet areas). The system aligns it with the SLAM map to generate a high-precision semantic map, marking no-entry areas (such as pet bowls) and priority cleaning areas (such as high-frequency contact surfaces). The layout map supports real-time updates (such as re-uploading after moving furniture) and is linked to historical cleaning records (such as "The area under the sofa was not cleaned last time" → automatically increase the priority of this area), providing dynamic environmental constraints for robot path planning to ensure the integrity and safety of cleaning coverage.

[0036] SLAM (Simultaneous Localization and Mapping) is a two-dimensional or three-dimensional environmental model constructed in real time by a robot in an unknown environment using sensors (lidar, cameras, etc.). It contains obstacle boundaries, traversable areas, and its own position information. The map is stored in the form of a grid or point cloud with a resolution of up to centimeters and supports dynamic updates (such as when new obstacles appear). SLAM technology enables autonomous navigation of robots and is widely used in cleaning robots, autonomous driving, and other fields, providing precise spatial references for path planning, obstacle avoidance, and task execution.

[0037] Step S4, prioritizing the work tasks and allocating resources, generating and sending a work plan to the cleaning robot; Wherein, in step S4, the following sub-steps are also included: S4-1, read the work task and verify its data integrity; S4-2, check whether there are conflicts between the subtasks of the work task. If so, adjust the task priority through LLM and generate a priority identifier; S4-3, check the hardware requirements of the subtasks in the work task, calculate the estimated time and energy consumption of each subtask, and generate a temporary charging plan through LLM if the total time exceeds the battery life; S4-4, generates a work plan based on the priority identification and temporary charging plan, and sends it to the cleaning robot through the MQTT protocol.

[0038] It should be noted that data integrity verification includes: JSON Schema validation: checks field types and required fields using predefined schemas (e.g., errors are reported if region coordinates are missing); Business rule engine: Verify logical rationality (e.g. suction value ≤ 5000Pa, water volume ≤ 1L / min); LLM semantic verification: This system uses a fine-tuned model to identify unusual task combinations (e.g., "carpet + high water volume" conflict). The system provides real-time error details (e.g., "Kitchen coordinates missing, please add them") and automatically corrects recoverable issues (e.g., completing default parameters). After verification, a task snapshot with a version number is generated to ensure data reliability during subsequent execution.

[0039] Adjusting task priorities means that LLM uses the priority model to dynamically score tasks based on real-time status (such as battery level and stain type) and historical data (such as user preferences and cleaning success rate). This includes: Input fusion: current environmental constraints (e.g., oil stains on the ground) + resource constraints (e.g., battery life 15%) + user habits (e.g., users often choose to skip bedroom cleaning); Rule reasoning: For example, the weight of the oil pollution task is increased by 20%, and the weight of non-critical tasks is reduced by -30% when the battery is insufficient; Output identification: Generate priority labels (such as P1: kitchen grease, P2: bedroom quick clean) and write them into the original task data; Conflict resolution: If P1 and P2 conflict, LLM automatically rearranges the order to ensure that the higher priority task is executed first.

[0040] Temporary charging solutions include: Breakpoint Marker: Insert a charging breakpoint into the task queue, recording the current coordinates, remaining battery power, and unfinished tasks (e.g., "Living Room 70% complete, battery 18%"); Recharging route: The Dijkstra algorithm is used to plan the shortest recharging route, reserving a safe amount of power (e.g., power ≥ 5%) to prevent power outages during recharging. Continue scanning and connection: After charging is completed, LLM regenerates the remaining task path (continues from the breakpoint coordinates). The plan is dynamically updated with the real-time power level and supports users to manually adjust the charging threshold.

[0041] Step S5: Establish a work path, drive the cleaning robot to execute the work plan, and perform cleaning work; Wherein, in step S5, the following sub-steps are also included: S5-1, the cleaning robot receives a work plan, which includes a work area, task execution priority, and charging point; S5-2, establish a SLAM map model and map the work area to the SLAM map model. If the work task involves multiple work areas, calculate the optimal area traversal order through LLM, use the hybrid A* algorithm to generate an S-shaped coverage path to obtain the work path; S5-3, the cleaning robot performs cleaning work along the working path.

[0042] It should be noted that calculating the optimal area traversal order through LLM means that when LLM receives multi-area tasks (such as living room, kitchen, bedroom) and real-time constraints (power, stain priority, user habits), it calls the multi-objective optimization model, assigns weights to the areas (such as the weight of the oily area is +30%, the high power consumption area is -20%, and the area frequently skipped by users is -50%), generates candidate sequences in descending order of weight, calculates the transfer distance between areas based on the SLAM map, and outputs the optimal traversal order (such as "kitchen→living room→bedroom") to ensure the coordinated optimization of cleaning efficiency, energy consumption balance and user preferences.

[0043] Use the hybrid A* algorithm to generate an S-shaped coverage path: The hybrid A* algorithm integrates the robot's kinematic constraint parameters (such as minimum turning radius, maximum speed, etc.) on the basis of the standard A algorithm, expands the motion nodes while evaluating the path length and turning cost. The algorithm rasterizes the working area, uses the S-shaped broken line as the heuristic target, and prioritizes the expansion of lateral coverage nodes; when encountering obstacles, it automatically inserts U-turns or detours to generate a continuous S-shaped path, and outputs the results as a sequence of coordinate points.

[0044] Step S6: The cleaning robot obtains status feedback in real time during the cleaning process and dynamically adjusts the task execution process according to the status feedback; Wherein, in step S6, the following sub-steps are also included: S6-1: During the cleaning process, the cleaning robot obtains real-time feedback on its own status and environmental status. The self-status feedback includes hardware status and task completion status. The hardware status includes the battery management system (BMS), motor current, water tank level, and roller brush speed. The task completion status includes task progress and path deviation. The environmental status feedback includes obtaining the coordinates of surrounding obstacles through the lidar, identifying the ground material and stain type through the visual module, and detecting abnormal noise through the sound sensor. S6-2, combining the cleaning robot's status feedback and historical abnormal data to perform status anomaly detection through LLM and identify abnormal states. Status feedback includes self-state feedback and environmental state feedback. Abnormal states include hardware abnormalities, task abnormalities, and environmental abnormalities. S6-3, LLM generates a dynamic task adjustment strategy based on the abnormal state. If multiple abnormal states exist, multi-objective optimization is called to generate adjustment priorities.

[0045] It should be noted that the laser radar obstacle coordinate acquisition and processing process includes: the cleaning robot is equipped with a 2D laser radar (such as RPLIDAR A3), which scans the surrounding environment information, obtains the polar coordinates (distance + angle) of the environmental information through the ToF principle, converts it into Cartesian coordinates through trigonometric functions, and forms a 360° point cloud (about 800 points / circle). Combined with the SLAM map, temporary noise points are eliminated to generate an obstacle list.

[0046] The visual module identifies the floor material and stain type through the following process: the robot is equipped with an RGB-D camera (such as Intel RealSense D435), and uses a lightweight MobileNetV3 model to extract texture features, classify materials such as hardwood, tiles, and carpets, and output stain masks through an improved U-Net segmentation network. It combines HSV color space thresholds to detect liquid / solid stains, output type labels (such as "coffee stains" and "pet hair") and confidence levels, and provide visual basis for dynamic adjustment of cleaning parameters (such as suction power and water volume).

[0047] Step S7: If the user changes the work task during the cleaning process, the system will immediately change the work plan based on the user's changes and status feedback; Wherein, in step S7, the following sub-steps are also included: S7-1, the user issues a voice command to the cleaning robot to change the work task; In S7-2, the cleaning robot captures user voice commands, transcribes them into text using automatic speech recognition (ASR), and uses the LLM to call the intent recognition model to analyze the user's change intentions. The user's change intentions include adding tasks, reducing tasks, and the number and type of tasks added or reduced. S7-3, LLM regenerates and sends the changed work plan to the cleaning robot based on the user's change intention and status feedback; S7-4, the cleaning robot performs cleaning work according to the modified work plan; In S7-5, the task management system sends the modified task plan to the mobile device via the wireless network, displays the modified task plan on the mobile app, and announces the change results through voice feedback TTS.

[0048] Step S8, storing status feedback, user instructions, work tasks, work history and change records in a database; Wherein, in step S8, the following sub-steps are also included: S8-1, establish a hierarchical database, which includes a time series database, a relational database, and a vector database; S8-2, storing state feedback via a time series database; The association between work tasks and user instructions is established through a relational database and stored in a table format. User instructions include key commands, mobile APP operation commands, and voice commands. The vector database stores work history and change records, including historical cleaning records and change history retained when users change work tasks; S8-3, when storing the user's voice command, the hierarchical database automatically detects and blocks personal identity information in the voice command through LLM sensitive word recognition.

[0049] It should be noted that the time series database is used to store high-frequency data generated in chronological order, such as the robot's power consumption, water volume, roller brush speed per second, etc. It supports fast writing and query by time range, and is suitable for monitoring the real-time operating status of robots and other equipment. Unlike traditional databases, the time series database optimizes storage and compression for time series, saving space and improving query speed. It is the core data source for fault warning and performance analysis.

[0050] Relational databases typically store structured data in tabular form, establish associations between tasks, user instructions, and execution results, and support transaction processing. When a user changes a task, all related tables are automatically updated synchronously to avoid data silos. In addition, relational databases also provide SQL query capabilities, facilitating statistics such as "areas most frequently skipped by users" and generating task completion reports. They are the basis for task management and user behavior analysis.

[0051] The vector database is used to convert work history and change records into computable semantic features, making it easier for LLM to quickly retrieve task types through similarity search and recommend optimal cleaning parameters. It supports semantic retrieval rather than keyword matching, achieving task optimization, abnormal pattern discovery and personalized recommendations, effectively supporting intelligent decision-making and continuous learning.

[0052] In addition, the hierarchical database also supports users to instantly change and delete stored user information through a visual interface.

[0053] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still make changes to the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These changes or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A cleaning robot task management method based on LLM, characterized in that: The method includes: Step S1: The user starts the cleaning robot task management system through instructions and enters the task management interface; Step S2: The user selects a work control through the task management interface; Step S3, the task management system generates a work task according to the user's selection of the work control; Step S4, prioritizing the work tasks and allocating resources, generating and sending a work plan to the cleaning robot; Step S5: Establish a work path, drive the cleaning robot to execute the work plan, and perform cleaning work; Step S6: The cleaning robot obtains status feedback in real time during the cleaning process and dynamically adjusts the task execution process according to the status feedback; Step S7: If the user changes the work task during the cleaning process, the system will immediately change the work plan based on the user's changes and status feedback; Step S8: storing status feedback, user instructions, work tasks, work history and change records in a database.

2. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, step S1 further includes the following sub-steps: S1-1, the user issues a voice start command to the cleaning robot, or starts the cleaning robot task management system by pressing a physical start button or a virtual start button on a mobile app, wherein the physical start button is located on the cleaning robot body, and the mobile app is installed on the user's mobile device, which is connected to the cleaning robot via a wireless network; S1-2, when the cleaning robot captures the voice start command, it transcribes the voice command into text through automatic speech recognition (ASR) to determine whether it contains the start keyword. If so, it starts the cleaning robot task management system; S1-3, when the cleaning robot task management system is started, the task management interface is displayed. The task management interface provides work controls through a visualization method, distinguishes the functional levels of the work controls, and determines the mapping relationship between the work controls and the robot's work tasks. The work tasks include cleaning areas, cleaning time, and cleaning modes.

3. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S2, the following sub-steps are also included: S2-1, the user sends a voice selection command to the cleaning robot, or selects the working control through the direct touch screen control method of the task management interface or the remote touch screen control method of the mobile terminal APP; S2-2, when the cleaning robot captures the user's voice selection command, it transcribes the voice selection command into text through automatic speech recognition (ASR), and uses the LLM to call the intent recognition model to parse the user's selection intention, and then automatically matches the work control options according to the user's selection intention; S2-3, after the user completes the selection, the task management interface and the mobile APP display the work control selected by the user and its work status.

4. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S3, the following sub-steps are also included: S3-1, the control system converts the user's selection of work controls into structured data and generates a structured task description, including work area coordinates, cleaning parameters, and constraints; S3-2, generating a work task based on the structured task description, historical cleaning records, and a room layout diagram, wherein the historical cleaning records are obtained from a database, and the room layout diagram is generated by user input in the task management interface; S3-3, decompose the work task into multiple subtasks, wherein the subtasks include basic tasks and combined tasks.

5. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S4, the following sub-steps are also included: S4-1, read the work task and verify its data integrity; S4-2, check whether there are conflicts between the subtasks of the work task. If so, adjust the task priority through LLM and generate a priority identifier; S4-3, check the hardware requirements of the subtasks in the work task, calculate the estimated time and energy consumption of each subtask, and generate a temporary charging plan through LLM if the total time exceeds the battery life; S4-4, generates a work plan based on the priority identification and temporary charging plan, and sends it to the cleaning robot through the MQTT protocol.

6. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S5, the following sub-steps are also included: S5-1, the cleaning robot receives the work plan, where the work plan includes a work area, a task execution priority, and a charging point; S5-2, establish a SLAM map model and map the work area to the SLAM map model. If the work task involves multiple work areas, calculate the optimal area traversal order through LLM, use the hybrid A* algorithm to generate an S-shaped coverage path to obtain the work path; S5-3, the cleaning robot performs cleaning work along the working path.

7. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S6, the following sub-steps are also included: S6-1: During the cleaning process, the cleaning robot obtains real-time feedback on its own status and environmental status. The self-status feedback includes hardware status and task completion status. The hardware status includes the battery management system (BMS), motor current, water tank level, and roller brush speed. The task completion status includes task progress and path deviation. The environmental status feedback includes obtaining the coordinates of surrounding obstacles through the lidar, identifying the ground material and stain type through the vision module, and detecting abnormal noise through the sound sensor. S6-2, combining the cleaning robot's state feedback and historical abnormality data to perform state anomaly detection through LLM to identify abnormal states, wherein the state feedback includes self-state feedback and environmental state feedback, and the abnormal states include hardware abnormalities, task abnormalities, and environmental abnormalities; S6-3, LLM generates a dynamic task adjustment strategy based on the abnormal state. If multiple abnormal states exist, multi-objective optimization is called to generate adjustment priorities.

8. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S7, the following sub-steps are also included: S7-1, the user issues a voice command to the cleaning robot to change the work task; S7-2, the cleaning robot captures the user's voice command, transcribes the voice command into text through automatic speech recognition (ASR), and uses the LLM to call the intent recognition model to analyze the user's change intention, which includes adding tasks, reducing tasks, and the number and type of tasks added or reduced; S7-3, LLM regenerates and sends the changed work plan to the cleaning robot based on the user's change intention and status feedback; S7-4, the cleaning robot performs cleaning work according to the modified work plan; In S7-5, the task management system sends the modified task plan to the mobile device via the wireless network, displays the modified task plan on the mobile app, and announces the change results through voice feedback TTS.

9. The cleaning robot task management method based on LLM according to claim 1, characterized in that: Wherein, in step S8, the following sub-steps are also included: S8-1, establishing a hierarchical database, wherein the hierarchical database includes a time series database, a relational database, and a vector database; S8-2, storing state feedback via a time series database; The association between work tasks and user instructions is established through a relational database and stored in a table format. User instructions include key commands, mobile APP operation commands, and voice commands. The vector database stores work history and change records, including historical cleaning records and change history retained when users change work tasks; S8-3, when storing the user's voice command, the hierarchical database automatically detects and shields personal identity information in the voice command through LLM sensitive word recognition.

Images