Interactive patrol robot agent system based on large language model

Abstract

This invention relates to the field of data processing, specifically to an interactive patrol robot intelligent agent system based on a large language model. The system includes a terminal interaction layer, a cloud decision-making layer, and a robot execution layer. The terminal interaction layer receives natural language commands or shortcut commands input by the user and sends them to the cloud decision-making layer. It also receives and displays task execution status and result information returned by the cloud decision-making layer. The cloud decision-making layer, deployed on a cloud server, receives user command information from the terminal interaction layer, robot status information and task execution result information from the robot execution layer, and pre-stored patrol scene map information. It uses a large language model to perform semantic understanding and logical reasoning on multi-source heterogeneous data, converting the user's natural language commands into standardized command data packages executable by the robot, and then sends them to the robot execution layer.

Description

Technical Field

[0001] This invention belongs to the field of data processing, specifically relating to an interactive patrol robot intelligent agent system based on a large language model. Background Technology

[0002] With the rapid development of artificial intelligence and robotics technologies, patrol robots are being used more and more widely in the security field. Traditional patrol robots mainly operate by patrolling along preset routes or through manual remote control. In preset route patrol mode, the robot can only patrol repeatedly according to a fixed route and time interval set in advance, and cannot flexibly adjust the patrol task according to changes in the actual scene and temporary needs of the user. In manual remote control mode, operators need to control the robot's movement and actions through complex control panels, joysticks, or keyboards and mice. This not only requires operators to have professional operating skills, but also easily leads to fatigue during long-term operation, resulting in low work efficiency and safety hazards.

[0003] Existing patrol robot systems generally suffer from unnatural human-computer interaction. Users cannot directly issue complex task commands to the robot using natural language (including text and voice); they can only select limited operation options through preset buttons or menus. When users need to perform non-standardized tasks, such as "check if there are any suspicious individuals loitering at the exit" or "check entrance 1A at 3 PM for tailgating and high passenger flow," traditional robot systems cannot understand the semantics of these natural language commands, let alone translate them into specific executable actions. Operators must manually break down these ambiguous natural language commands into a series of precise machine instructions, including moving the target point, performing actions, sensor activation status, and feedback content, which greatly increases the complexity and time cost of operation.

[0004] Furthermore, traditional patrol robots typically utilize embedded processors with very limited computing and storage capabilities. This prevents them from processing complex, multi-source, heterogeneous data locally, such as real-time analysis of high-definition video streams, processing of LiDAR point cloud data, semantic understanding of natural language, and complex task planning. Existing patrol systems lack an intelligent decision-making center capable of integrating user intent, environmental map information, robot status information, and real-time perception data. During task execution, robots can only mechanically perform operations according to preset programs, unable to make autonomous decisions or dynamically adjust based on environmental changes and task progress. When encountering situations outside the preset program, the robot often becomes stuck, requiring manual intervention to continue working, which severely impacts patrol efficiency and the system's intelligence level.

[0005] In terms of task execution, existing patrol robot systems have a coarse-grained approach. The systems can only perform simple, standardized tasks such as fixed-point patrols, timed patrols, and anomaly alarms. For vague user commands, the systems cannot automatically parse key information such as time requirements, location requirements, specific actions, required sensor functions, and the type of feedback. This often results in robots failing to accurately meet the user's actual needs, and may even lead to task execution errors. For example, when a user commands "Go check warehouse area B," a traditional robot might simply move to warehouse area B and then return, without actively activating its camera for video monitoring, detecting any anomalies, or providing feedback to the user.

[0006] To address these issues, some existing technologies attempt to apply artificial intelligence to patrol robots. For example, some patent documents disclose patrol robot control systems based on voice recognition technology. These systems can recognize simple voice commands from users, such as "forward," "turn left," and "stop," but they still cannot understand complex natural language commands or perform autonomous task decomposition and planning. Other patent documents disclose patrol robot anomaly detection systems based on deep learning. These systems can detect anomalies in the environment through cameras and sensors, but they still lack the ability to interact naturally with users and make intelligent decisions.

[0007] In summary, existing patrol robot systems suffer from problems such as unnatural human-computer interaction, high operational barriers, lack of an intelligent decision-making center, inability to autonomously decompose complex tasks, coarse task execution granularity, and low level of intelligence. These shortcomings make it difficult to meet the requirements of modern security patrol scenarios for flexibility, intelligence, and efficiency in robot systems. Therefore, there is an urgent need to develop an interactive patrol robot intelligent agent system capable of natural language interaction, autonomous task decomposition and planning, multi-source information fusion decision-making, and closed-loop task execution and feedback. Summary of the Invention

[0008] The purpose of this invention is to provide an interactive patrol robot intelligent agent system based on a large language model to solve the problems mentioned in the background art.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0010] An interactive patrol robot intelligent agent system based on a large language model includes the following steps:

[0011] It includes a terminal interaction layer, a cloud decision-making layer, and a robot execution layer. The terminal interaction layer communicates with the cloud decision-making layer via a wireless network, and the cloud decision-making layer communicates with the robot execution layer via a wireless network.

[0012] The terminal interaction layer is used to receive natural language commands or shortcut commands input by the user and send them to the cloud decision layer. It is also used to receive and display the task execution status and result information returned by the cloud decision layer.

[0013] The cloud-based decision-making layer is deployed on a cloud server and is used to receive user instruction information from the terminal interaction layer, robot status information and task execution result information from the robot execution layer, as well as pre-stored patrol scene map information. It performs semantic understanding and logical reasoning on multi-source heterogeneous data through a large language model, converts the user's natural language instructions into standardized instruction data packets that can be executed by the robot, and sends them to the robot execution layer.

[0014] The robot execution layer is the autonomous mobile robot body, which is used to receive standardized instruction data packets issued by the cloud decision layer, parse them, execute corresponding patrol tasks, and report the robot's own status information, environmental data collected by sensors, and analysis results to the cloud decision layer in real time.

[0015] Furthermore, the terminal interaction layer includes a dialogue interaction module and a shortcut command module. The dialogue interaction module is used to receive text or voice commands input by the user, convert the voice commands into text commands, and send them to the cloud decision-making layer. It is also used to receive feedback information in text or voice form returned by the cloud decision-making layer and present it to the user through the display screen and speaker. The shortcut command module includes multiple graphical touch buttons, each of which is associated with a preset patrol task shortcut command. When the user clicks a graphical touch button, the shortcut command module automatically generates the corresponding standardized shortcut command data package and sends it to the cloud decision-making layer.

[0016] Furthermore, the cloud-based decision-making layer includes a data receiving interface, a large language model processing unit, and an instruction structured output unit. The data receiving interface is communicatively connected to both the terminal interaction layer and the robot execution layer, and is used to receive user instruction information from the terminal interaction layer, robot local status information, current task execution status information, and environmental data and AI analysis results collected by sensors from the robot execution layer. It is also used to receive pre-stored patrol scene map information. The large language model processing unit is loaded with a pre-trained large language model, and its input is connected to the data receiving interface. It is used to fuse and process the received multi-source heterogeneous data, perform semantic understanding of the user's natural language instructions, and perform logical reasoning by combining patrol scene map information and robot current status information to decompose the task's time requirements, location requirements, action requirements, functional requirements, and feedback requirements. The input of the instruction structured output unit is connected to the output of the large language model processing unit, and is used to convert the reasoning results output by the large language model processing unit into standardized instruction data packets that can be executed by the robot, and then send them to the robot execution layer via a wireless network.

[0017] Furthermore, the standardized instruction data packet includes at least a time field, a location field, a task field, a function field, and a feedback field; the time field is used to specify the execution time of the task, including three types: immediate execution, scheduled execution, and periodic execution.

[0018] Furthermore, the location field specifies the execution location of the task, including the geographic coordinates of the target point, the area range, and path planning requirements; the task field specifies the specific action type that the robot needs to perform, including navigation movement, stationary stop, rotational scanning, following movement, and emergency alarm; the function field specifies the sensors and analysis functions that the robot needs to activate during task execution, including camera video acquisition, LiDAR obstacle avoidance, smoke detection, temperature detection, sound detection, face recognition, and abnormal behavior analysis; the feedback field specifies the data type of results that the robot needs to send back to the cloud decision-making layer, including real-time location information, battery information, speed information, raw sensor data, AI analysis results, video streams, and images.

[0019] Furthermore, the robot execution layer includes a chassis movement mechanism, a sensor suite, an execution controller, and a status feedback module. The chassis movement mechanism uses a wheeled or tracked chassis to enable the robot to move, turn, and stop according to the instructions of the execution controller. The sensor suite includes at least a camera, lidar, microphone array, smoke sensor, temperature sensor, and infrared sensor to collect image data, point cloud data, sound data, smoke concentration data, temperature data, and infrared radiation data in the patrol environment. The execution controller communicates with the cloud decision-making layer via a wireless communication module to receive standardized instruction data packets from the cloud decision-making layer, parses the instruction data packets, generates corresponding control signals, and controls the chassis movement mechanism and sensor suite to perform corresponding tasks. The status feedback module is connected to the chassis movement mechanism, sensor suite, and execution controller to collect the robot's local status information in real time, including battery level, position, speed, attitude, fault information, environmental data collected by the sensor suite, and AI analysis results, and reports this information to the cloud decision-making layer in real time via a wireless network.

[0020] Furthermore, the large language model processing unit also includes an instruction error correction module and a task scheduling module. The instruction error correction module is used to perform grammatical error correction and semantic completion on the natural language instructions input by the user. When the instructions input by the user contain grammatical errors, semantic ambiguity, or incomplete information, it obtains the missing key information through multi-turn dialogue interaction with the user to ensure the accuracy and completeness of the instructions. The task scheduling module is used to reasonably schedule and sort multiple tasks according to the robot's current state, task priority, and urgency. When multiple tasks are issued at the same time, the task with higher urgency is executed first, and the execution time of other tasks is reasonably arranged to avoid task conflicts.

[0021] Furthermore, the cloud-based decision-making layer also includes a map management module and a data storage module. The map management module is used to store and manage high-precision electronic maps of the patrol scenario, including map creation, updating, deletion, and query functions. It also supports real-time map updates, ensuring the accuracy of robot navigation by updating map information promptly when the patrol scenario changes. The data storage module is used to store user command history, task execution records, sensor data, AI analysis results, and system log data, providing data support for subsequent data analysis, system optimization, and accident tracing.

[0022] Furthermore, the robot execution layer also includes a local emergency processing module, which is connected to the execution controller. When communication between the robot and the cloud decision layer is interrupted, the module automatically switches to local emergency mode and executes preset emergency tasks, including returning to the charging station, standing by in place, and issuing audible and visual alarms, to ensure the robot's safety in the event of a communication interruption.

[0023] Furthermore, the terminal interaction layer also includes a task monitoring module, which is used to display the robot's current position, task execution status, sensor data, and video stream in real time.

[0024] This application also discloses an electronic device, including:

[0025] At least one processor; and

[0026] A memory communicatively connected to the at least one processor; wherein,

[0027] The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to execute the interactive patrol robot intelligent agent system based on a large language model of the present invention.

[0028] Beneficial effects: By introducing a large language model, this invention allows users to directly issue complex task commands to the robot using natural language (text or voice) without needing specialized operational skills. The system can accurately understand the user's natural language intent and translate it into commands that the robot can execute, greatly simplifying the operation process, lowering the operational threshold, and improving the naturalness of human-computer interaction and user experience.

[0029] This invention employs a three-layer distributed architecture of terminal-cloud-robot, transferring complex computational tasks (including natural language understanding, semantic reasoning, and task planning) to the cloud server for execution. This fully leverages the powerful computing and storage capabilities of the cloud, compensating for the robot's insufficient computing power. The cloud-based decision layer can integrate multi-source heterogeneous information such as user commands, environmental maps, robot status, and sensor data for comprehensive analysis and logical reasoning. It autonomously decomposes fuzzy user commands into precise machine instructions, eliminating the need for manual path programming and achieving autonomous task planning.

[0030] This invention transforms the reasoning results of a large language model into a standardized instruction data package containing key information such as time, location, action, function, and feedback through a structured instruction output unit. This enables the robot to accurately understand the specific requirements of the task and execute fine-grained task actions. Simultaneously, the system acquires the robot's execution status and sensor data in real time through a status feedback module, achieving closed-loop task execution and feedback. This ensures that the task is completed accurately according to the user's requirements, improving the accuracy and efficiency of task execution.

[0031] The system boasts a clear structure, rapid response, and high reliability: This invention employs a three-layer distributed architecture with clearly defined responsibilities and standardized interfaces, facilitating system development, maintenance, and expansion. Communication via standardized instruction data packets ensures accurate instruction parsing and efficient transmission. Furthermore, the system possesses local emergency handling capabilities; in the event of communication interruption, the robot can automatically execute pre-set emergency tasks, ensuring system safety and reliability.

[0032] It possesses powerful information fusion capabilities and decision-making accuracy: The cloud-based decision-making layer can receive multi-source information in real time from the terminal interaction layer, robot execution layer, and map management module, and perform comprehensive analysis and processing to make more accurate and reasonable decisions. For example, when the robot is performing patrol tasks, the system can combine real-time environmental data and robot status information to dynamically adjust the patrol path and task priority, improving the effectiveness and targeting of the patrol.

[0033] Support for multi-task scheduling and dynamic adjustment: The large language model processing unit of this invention includes a task scheduling module, which can reasonably schedule and sort multiple tasks according to their priority and urgency. When an urgent task is issued, the system can pause the currently executing non-urgent task, prioritize the execution of the urgent task, and resume the execution of the paused task after the urgent task is completed, thereby improving the system's task processing capability and response speed. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the interface of the terminal interaction layer of the present invention;

[0035] Figure 2 This is a flowchart of the instruction processing and execution process of the system of the present invention. Detailed Implementation

[0036] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0037] refer to Figure 1-2 This invention provides an interactive patrol robot intelligent agent system based on a large language model, employing a three-layer distributed architecture consisting of a terminal interaction layer 100, a cloud decision-making layer 200, and a robot execution layer 300. The terminal interaction layer 100 communicates with the cloud decision-making layer 200 via a 4G / 5G or Wi-Fi wireless network, and the cloud decision-making layer 200 communicates with the robot execution layer 300 via a 4G / 5G or Wi-Fi wireless network.

[0038] The terminal interaction layer 100 is deployed on the user's mobile terminal (such as a smartphone or tablet) or computer, and is presented in the form of a mobile application (App) or a web interface. The terminal interaction layer 100 is mainly responsible for interacting with the user, receiving the user's command input, and presenting the result information returned by the cloud decision layer 200 to the user.

[0039] The cloud-based decision-making layer 200, deployed on a high-performance cloud server cluster, serves as the intelligent brain of the entire system. It receives and processes all data from the terminal interaction layer 100 and the robot execution layer 300, performing semantic understanding, logical reasoning, and task planning through a large language model to generate standardized instruction data packets, which are then distributed to the robot execution layer 300. Simultaneously, the cloud-based decision-making layer 200 is also responsible for managing patrol scene maps, storing system data, and performing task scheduling.

[0040] The robot execution layer 300 consists of the autonomous mobile robot itself, deployed in actual patrol scenarios. The robot execution layer 300 is responsible for receiving standardized instruction data packets from the cloud-based decision layer 200, parsing them, executing the corresponding patrol tasks, and reporting its own status information, sensor-collected environmental data, and analysis results to the cloud-based decision layer 200 in real time.

[0041] The specific structure and function of each layer are described in detail below:

[0042] The terminal interaction layer 100 includes a dialogue interaction module 101, a shortcut command module 102, and a task monitoring module 103.

[0043] The dialogue interaction module 101 is the main interface for users to interact with the system using natural language. The dialogue interaction module 101 includes a text input box, a voice input button, a dialogue display area, and voice playback functionality. Users can input text commands through the text input box or voice commands through the voice input button. When a user inputs a voice command, the dialogue interaction module 101 first converts the voice signal into a text signal using speech recognition technology, and then sends the text command to the cloud decision layer 200. Simultaneously, the dialogue interaction module 101 receives text feedback information returned from the cloud decision layer 200, displays it in the dialogue display area, and can convert the text information into a voice signal using speech synthesis technology, which is then played back to the user through a speaker.

[0044] For example, a user can input via voice: "Railway Police No. 1, go to entrance 1A at 3 PM to check for tailgating and high passenger flow, and send the video back." The dialogue interaction module 101 converts this voice command into a text command and sends it to the cloud decision layer 200. When the cloud decision layer 200 processes the command and returns the result, the dialogue interaction module 101 will display in the dialogue display area: "Okay, I will go to entrance 1A at 3 PM to check for tailgating and high passenger flow, and send back the video stream in real time." Simultaneously, this information will be played back via voice.

[0045] The shortcut command module 102 is located at the bottom of the terminal interface and contains multiple graphical touch buttons, each associated with a preset patrol task shortcut command. These shortcut commands are pre-set based on common security patrol tasks, such as "one-click patrol," "emergency alarm," "follow mode," "return to charging," and "fixed-point monitoring." When a user clicks a shortcut button, the shortcut command module 102 automatically generates the corresponding standardized shortcut command data package and sends it to the cloud decision layer 200. The design of the shortcut command module 102 allows users to quickly issue frequently used standard tasks, improving operational efficiency.

[0046] For example, when a user clicks the "One-Click Patrol" button, the shortcut command module 102 generates a standardized command data packet containing the preset patrol route, patrol time, sensor on / off status, and feedback requirements, and sends it to the cloud decision layer 200. Upon receiving the command, the cloud decision layer 200 immediately issues it to the robot execution layer 300, and the robot begins patrolling according to the preset route.

[0047] The task monitoring module 103 is used to monitor the robot's task execution status in real time. The interface of the task monitoring module 103 includes an electronic map display area, a robot status display area, and a video monitoring area. The electronic map display area shows the robot's current location, patrol route, and task target point in real time. The robot status display area displays information such as the robot's battery level, speed, posture, currently executing task name, and task progress. The video monitoring area displays the real-time video stream captured by the robot's camera. Users can view the patrol task execution status in real time through the task monitoring module 103 and can pause, resume, or cancel the currently executing task at any time using buttons on the interface.

[0048] For example, when the robot is performing tailgating and high-passenger-flow detection tasks at entrance 1A, the user can see the robot moving from the charging station along a planned path towards entrance 1A on the electronic map of the task monitoring module 103. Simultaneously, the user can see real-time footage captured by the robot's camera in the video monitoring area, and information such as the robot's battery level, speed, and task progress can be displayed in the robot status display area. If the user discovers an emergency requiring immediate robot intervention, they can click the "Emergency Alarm" button, and the system will immediately interrupt the current task to execute the emergency alarm task.

[0049] The cloud-based decision-making layer 200 includes a data receiving interface 201, a large language model processing unit 202, an instruction structured output unit 203, a map management module 204, a data storage module 205, and a task scheduling module 206.

[0050] The data receiving interface 201 serves as the interface for data interaction between the cloud-based decision-making layer 200 and external systems. The data receiving interface 201 communicates with the terminal interaction layer 100 and the robot execution layer 300 via wireless networks, and also connects to the map management module 204 and the data storage module 205. The data receiving interface 201 is responsible for receiving user command information from the terminal interaction layer 100, robot local status information, current task execution status information, environmental data collected by sensors, and AI analysis results from the robot execution layer 300, as well as patrol scene map information from the map management module 204. After formatting and preprocessing all received data, the data receiving interface 201 sends it to the large language model processing unit 202.

[0051] The large language model processing unit 202 is the core module of the cloud decision layer 200, loaded with a finely tuned pre-trained large language model. In this embodiment, a large language model based on the Transformer architecture is used. This model has been pre-trained on a large-scale text corpus and finely tuned for professional terminology and task scenarios in the field of security patrol to improve the understanding and reasoning accuracy of security patrol-related instructions.

[0052] The input end of the large language model processing unit 202 is connected to the data receiving interface 201 to receive preprocessed multi-source heterogeneous data. The large language model processing unit 202 first performs semantic understanding on the user's natural language commands, extracting key information from the commands, including task type, execution time, execution location, action requirements, functional requirements, and feedback requirements. Then, the large language model processing unit 202 combines the robot's current state information (such as location, battery level, and whether it is performing other tasks) and patrol scene map information obtained from the data receiving interface 201 to perform logical reasoning and task planning, determining the specific execution plan for the task.

[0053] For example, when the large language model processing unit 202 receives the user instruction "Go to entrance 1A at 3 PM to check for tailgating and high passenger flow, and send the video back," it first performs semantic understanding and extracts the following key information:

[0054] Execution time: 3:00 PM (3:00 PM);

[0055] Location of execution: Entrance 1A;

[0056] Task type: Tail-in entry and high passenger flow detection;

[0057] Functional requirement: Enable camera;

[0058] Feedback request: Return structured images and real-time video streams of passengers following behind and those with high passenger flow;

[0059] Then, the large language model processing unit 202 queries the robot's current status and learns that the robot is currently at the charging station with 95% battery and no ongoing tasks. Next, the large language model processing unit 202 obtains the coordinates (X: 123.456, Y: 78.901) of entrance 1A and the optimal path from the charging station to entrance 1A from the map management module 204. Finally, the large language model processing unit 202 generates a task execution plan: the robot departs from the charging station at 15:00, navigates along the planned path to entrance 1A, and upon arrival, activates its camera to perform tail-entry and high-passenger-flow detection, while simultaneously transmitting structured images and real-time video streams of tail-entry and high-passenger-flow detection back to the cloud decision layer 200.

[0060] The large language model processing unit 202 also includes an instruction correction module 2021. The instruction correction module 2021 is used to perform grammatical correction and semantic completion on the natural language instructions input by the user. When the user-input instruction contains grammatical errors, semantic ambiguity, or incomplete information, the instruction correction module 2021 generates clarification questions and sends them to the terminal interaction layer 100 through the data receiving interface 201. This facilitates multi-round dialogue with the user to obtain missing key information, ensuring the accuracy and completeness of the instruction.

[0061] For example, when a user inputs the command "Go check the warehouse," the command correction module 2021 will recognize that the command information is incomplete, lacking a specific warehouse location and the action to be performed. Therefore, the command correction module 2021 will generate a clarification question: "Which warehouse do you want to check? What do you need me to check?" and send it to the terminal interaction layer 100. After the user replies "Go to warehouse B area to see if there are any strangers," the command correction module 2021 will send the completed command to the large language model processing unit 202 for further processing.

[0062] The input of the instruction structured output unit 203 is connected to the output of the large language model processing unit 202, and is used to convert the task execution plan in natural language form output by the large language model processing unit 202 into a standardized instruction data packet that can be executed by the robot. The standardized instruction data packet adopts JSON format, which has the characteristics of clear structure, easy parsing, and high transmission efficiency.

[0063] In this embodiment, the standardized instruction data packet includes at least the following fields:

[0064] task_id: A unique identifier for a task, used to distinguish different tasks;

[0065] time: The time field specifies the execution time of the task. The format is "YYYY-MM-DD HH:MM:SS". If it is to be executed immediately, the value of this field is "immediate"; if it is to be executed periodically, the value of this field is "periodic" with an additional period parameter, such as "periodic: 3600" which means to execute once per hour.

[0066] `location`: This field specifies the location where the task will be executed. It includes the coordinates (x, y) of the target point and the area range. If it is a path patrol, it includes the coordinates of multiple points along the path.

[0067] action: The task field specifies the specific action type that the robot needs to perform.

[0068] Optional values ​​include "navigate", "stay", "rotate", "follow", and "alarm".

[0069] `function`: This field specifies the sensors and analysis functions that the robot needs to activate during task execution. It is an array, with possible values ​​including "camera", "lidar", "microphone", "smoke sensor", "temperature sensor", "face recognition", and "abnormal behavior analysis", etc.

[0070] Feedback: This field specifies the type of results the robot needs to send back to the cloud-based decision-making layer. It is an array, with possible values ​​including "position" (real-time position), "battery" (battery charge), "speed" (speed), "sensor_data" (raw sensor data), "ai_result" (AI analysis results), "video_stream" (video stream), and "image" (image).

[0071] priority: Task priority, used for task scheduling. Optional values ​​are 1-5, where 1 is the lowest priority and 5 is the highest priority.

[0072] The instruction structured output unit 203 sends the generated standardized instruction data packet to the robot execution layer 300 through the data receiving interface 201.

[0073] The map management module 204 is used to store and manage high-precision electronic maps of the patrol scene. The map management module 204 supports multiple map formats, including raster maps, vector maps, and point cloud maps. The map management module 204 provides map creation, updating, deletion, and query functions. Users can upload map data of the patrol scene through the terminal interaction layer 100, or perform real-time mapping using the LiDAR of the robot execution layer 300. When the patrol scene changes, users can update the map information through the terminal interaction layer 100, and the map management module 204 will synchronize the updated map to the large language model processing unit 202 and the robot execution layer 300 to ensure the accuracy of robot navigation.

[0074] The data storage module 205 employs a distributed database system to store all data generated during system operation, including user command history, task execution records, sensor data, AI analysis results, system logs, and map data. The data storage module 205 supports fast data writing and querying, and provides data backup and recovery functions to ensure data security and integrity. The stored data can be used for subsequent data analysis, system optimization, incident tracing, and report generation.

[0075] The task scheduling module 206, connected to the large language model processing unit 202 and the instruction structured output unit 203, is used to rationally schedule and prioritize multiple tasks. When multiple tasks are issued simultaneously, the task scheduling module 206 sorts and schedules the tasks according to their priority, urgency, execution time requirements, and the robot's current state. The task scheduling module 206 employs a priority-based preemptive scheduling algorithm. When a high-priority urgent task is issued, the currently executing low-priority task is immediately paused, and the high-priority task is executed first. After the high-priority task is completed, the paused low-priority task resumes execution.

[0076] For example, while the robot is performing a routine patrol task with priority 3, the user issues an emergency alarm task with priority 5. The task scheduling module 206 will immediately send an instruction to the robot execution layer 300 to pause the current task, and then send the emergency alarm task instruction. Upon receiving the instruction, the robot will immediately stop the current patrol task and proceed to the alarm location to execute the emergency alarm task. After the emergency alarm task is completed, the task scheduling module 206 will send an instruction to resume the previously paused patrol task, and the robot will continue to complete the remaining patrol tasks.

[0077] The robot execution layer 300 is the autonomous mobile robot body, including a chassis movement mechanism 301, a sensor kit 302, an execution controller 303, a status feedback module 304, and a local emergency processing module 305.

[0078] The chassis mobility mechanism 301 employs a four-wheel differential drive wheeled chassis, suitable for flat indoor and outdoor patrol scenarios. For patrol scenarios with complex terrain, a tracked chassis or a Mecanum wheel chassis can also be used. The chassis mobility mechanism 301 includes a drive motor, a reducer, wheels, and a suspension system. The actuator controller 303 controls the speed and direction of the drive motor to achieve the robot's forward, backward, left, right, and stationary rotation movements. The chassis mobility mechanism 301 is also equipped with a braking system and collision avoidance sensors to ensure the safety of the robot during movement.

[0079] The sensor suite 302 is an important component of the robot's perception of the environment, and includes at least a camera 3021, a lidar 3022, a microphone array 3023, a smoke sensor 3024, a temperature sensor 3025, and an infrared sensor 3026.

[0080] Camera 3021 is a high-definition network camera mounted on the top of the robot, capable of 360-degree rotation. Camera 3021 is used to collect video image data from the patrol environment and transmit it to the execution controller 303 for AI processing such as face recognition, abnormal behavior analysis, and license plate recognition.

[0081] The LiDAR 3022 is a two-dimensional LiDAR installed in the middle of the robot. It has a scanning frequency of 10Hz and a ranging range of 0.1-20 meters. The LiDAR 3022 is used to collect point cloud data of the environment, enabling the robot's autonomous navigation, obstacle avoidance, and mapping functions.

[0082] The microphone array 3023 consists of four microphones mounted on top of the robot to collect sound data from the environment. The microphone array 3023 can directionally locate and amplify sound, and is used to detect abnormal sounds such as breaking glass, explosions, and cries for help.

[0083] The smoke sensor 3024 is a photoelectric smoke sensor mounted on the top of the robot to detect the smoke concentration in the environment. When the smoke concentration exceeds a set threshold, the smoke sensor 3024 sends an alarm signal to the actuator controller 303.

[0084] Temperature sensor 3025 is a digital temperature sensor mounted on the side of the robot to detect the ambient temperature. When the temperature exceeds a set threshold, temperature sensor 3025 sends an alarm signal to actuator controller 303.

[0085] The 3026 infrared sensor is a passive infrared sensor that is installed around the robot to detect infrared radiation from the human body, thus enabling personnel detection.

[0086] The execution controller 303 employs a high-performance embedded ARM processor and runs a real-time operating system. The execution controller 303 communicates with the cloud-based decision layer 200 via a wireless communication module (4G / 5G or Wi-Fi). The main functions of the execution controller 303 include:

[0087] Receive standardized instruction data packets issued by the cloud-based decision-making layer 200, and parse the instruction data packets;

[0088] Based on the parsed instructions, corresponding control signals are generated to control the chassis moving mechanism 301 and sensor kit 302 to perform corresponding tasks;

[0089] The raw data collected by sensor kit 302 is preprocessed and analyzed using AI to generate analysis results;

[0090] It communicates with the status feedback module 304 to receive the robot's status information and sensor data;

[0091] It communicates with the local emergency response module 305 to handle abnormal situations such as communication interruption.

[0092] The status feedback module 304 is connected to the chassis moving mechanism 301, the sensor kit 302, and the execution controller 303. The status feedback module 304 is used to collect the robot's local status information and sensor data in real time. The local status information includes the robot's battery level, position, speed, posture, motor temperature, and fault information. The sensor data includes video image data collected by the camera, point cloud data collected by the lidar, sound data collected by the microphone array, smoke concentration data collected by the smoke sensor, temperature data collected by the temperature sensor, and human detection data collected by the infrared sensor.

[0093] The status feedback module 304 packages and processes all collected data, and then reports it in real time to the cloud decision layer 200 through the wireless communication module of the execution controller 303. The reporting frequency can be adjusted according to the requirements of the task. For tasks with high real-time requirements, such as emergency alarm tasks, the reporting frequency can be set to 10Hz; for routine patrol tasks, the reporting frequency can be set to 1Hz.

[0094] The local emergency handling module 305 is connected to the execution controller 303 to handle abnormal situations encountered by the robot during operation, especially communication interruptions with the cloud decision layer 200. The local emergency handling module 305 has multiple built-in emergency handling plans. When a communication interruption is detected to exceed a set time threshold (e.g., 30 seconds), the local emergency handling module 305 will automatically trigger the corresponding emergency handling plan.

[0095] In this embodiment, the local emergency response module 305 provides the following emergency response modes:

[0096] Return to charging mode: The robot automatically stops the current task and autonomously navigates back to the charging station to charge according to the pre-stored map information;

[0097] Standby mode: The robot stops its current task and stands by in place, while activating the sound and light alarm to alert people in the vicinity;

[0098] Continue execution mode: The robot continues to execute the current task until the task is completed or communication is restored;

[0099] Safe Zone Mode: The robot automatically navigates to a pre-set safe zone and waits in place.

[0100] Users can pre-set emergency handling modes for communication interruptions through the terminal interaction layer 100. When communication is restored, the local emergency handling module 305 will report the robot's status information and task execution status during the communication interruption to the cloud decision layer 200. The cloud decision layer 200 will decide whether to continue executing the previous task or issue a new task based on the actual situation.

[0101] The workflow of the system of the present invention will be explained in detail below with reference to a specific application scenario:

[0102] Application Scenario: The security patrol system of a factory has deployed the interactive patrol robot intelligent agent system based on a large language model, as described in this invention. The patrol area of ​​the factory includes the production workshop, warehouse, office building, and the perimeter of the factory. The system has a pre-stored high-precision electronic map of the factory, and the robot is on standby at the charging station during normal operation.

[0103] Workflow:

[0104] The user issued the following instruction: At 2:50 PM, a railway police officer input a voice command through the terminal app on his mobile phone: "Railway Police No. 1, go to entrance 1A at 3 PM to check for any tailgating or high passenger flow, and send the video back."

[0105] Terminal interaction layer processing: The dialogue interaction module 101 of the terminal App converts voice commands into text commands, and then sends them to the data receiving interface 201 of the cloud decision layer 200 via wireless network.

[0106] Cloud data reception and preprocessing: After receiving the user's text instructions, the data reception interface 201 of the cloud decision layer 200 performs format unification and preprocessing, and then sends them to the large language model processing unit 202.

[0107] Large Language Model Semantic Understanding and Reasoning: The large language model processing unit 202 performs semantic understanding of the user's text commands, extracting key information: execution time is 15:00, execution location is entrance 1A, the task is tailgating into the station and high passenger flow detection, the functional requirement is to activate the camera, and the feedback requirement is to return structured images and real-time video streams of tailgating into the station and high passenger flow. Then, the large language model processing unit 202 queries the robot's current status, learning that the robot is located at the charging station, its battery is at 95%, and it has no tasks currently being executed. Next, the large language model processing unit 202 obtains the coordinates of entrance 1A (X: 123.456, Y: 78.901) and the optimal path from the charging station to entrance 1A from the map management module 204. Finally, the large language model processing unit 202 generates a task execution plan.

[0108] Structured instruction output: The structured instruction output unit 203 converts the task execution plan output by the large language model processing unit 202 into a standardized instruction data packet in JSON format, and sends it to the execution controller 303 of the robot execution layer 300 through the data receiving interface 201.

[0109] The robot receives and parses instructions: After receiving the standardized instruction data packet, the execution controller 303 of the robot execution layer 300 parses the instructions and extracts the information of each field.

[0110] Task execution preparation: Execution controller 303 sets the task execution timer according to the time field in the instruction. Simultaneously, execution controller 303 initializes the camera and prepares for task execution.

[0111] Task Execution: When 15:00 arrives, the execution controller 303 sends navigation instructions to the chassis moving mechanism 301, controlling the robot to start from the charging pile and move along the planned path towards entrance 1A. During the movement, the status feedback module 304 collects real-time status information such as the robot's position, battery level, and speed, and reports this information to the cloud decision layer 200 via the execution controller 303. The cloud decision layer 200 then sends this status information to the task monitoring module 103 of the terminal app, allowing railway police officers to view the robot's position and status in real time.

[0112] Upon reaching the target point, the robot performs the detection task: After reaching the target point at entrance 1A, the execution controller 303 controls the chassis moving mechanism 301 to stop moving. Then, the execution controller 303 sends an activation command to the camera 3021 to begin acquiring real-time video streams.

[0113] Data Acquisition and Analysis: Camera 3021 acquires real-time video streams, which are then analyzed by controller 303 to determine whether tailgating or high passenger flow events have occurred, generating structured images. Simultaneously, the structured images of tailgating and high passenger flow events, along with the real-time video stream acquired by camera 3021, are transmitted in real-time to the cloud decision layer 200 via the wireless communication module of controller 303.

[0114] Abnormal situation handling: If the camera detects tailgating or high passenger flow events, the execution controller 303 will immediately generate an alarm message of "suspected tailgating / high passenger flow" and report it to the cloud decision layer 200 through the status feedback module 304.

[0115] Cloud-based alarm information processing: After receiving the alarm information, the data receiving interface 201 of the cloud decision layer 200 sends it to the large language model processing unit 202. The large language model processing unit 202 generates an alarm information in natural language: "Suspected tailgating / high passenger flow detected at entrance 1A, please handle immediately!" At the same time, the large language model processing unit 202 generates an emergency task instruction, requiring the robot to perform a 360-degree rotation scan at entrance 1A to collect more video and image information.

[0116] Alarm Information Push: The cloud-based decision layer 200 sends the generated alarm information and the real-time video stream collected by the robot to the dialogue interaction module 101 and task monitoring module 103 of the terminal app. The dialogue interaction module 101 sends alarms to the railway police station officers through voice broadcast and text display. The task monitoring module 103 displays the robot's rotation scanning process and video footage in real time.

[0117] Task Completion and Feedback: After the robot completes the tasks of tailgating into the station and detecting high passenger flow, the execution controller 303 generates a task completion report, which is then reported to the cloud decision layer 200 via the status feedback module 304. The large language model processing unit 202 of the cloud decision layer 200 generates a task completion report in natural language: "The tasks of tailgating into the station and detecting high passenger flow at entrance 1A have been completed. The detection time was from 15:00 to 15:05 on March 12, 2026. The detection result was that suspected tailgating into the station / high passenger flow was detected, and an alarm message has been pushed. The robot's current battery level is 88%, and it is returning to the charging station." Then, the cloud decision layer 200 sends the task completion report to the terminal App, which railway police officers can view the task completion status.

[0118] Robot returns to charging: After completing the task, the controller 303 controls the robot to autonomously navigate back to the charging station for charging, in preparation for the next task.

[0119] As can be seen from the above workflow, the system of the present invention can realize natural language interaction, autonomous task decomposition and planning, multi-source information fusion decision-making, and closed-loop task execution and feedback, which greatly improves the intelligence level and work efficiency of the patrol robot system.

[0120] This application also provides an embodiment of an electronic device. The electronic device is manifested in the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: one or more processors or processing units, memory, and buses connecting different components (including memory and processing units).

[0121] A bus refers to one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. Examples of these architectures include, but are not limited to, Industry Standard Architecture (ISA) buses, Micro Channel Architecture (MCA) buses, Enhanced ISA buses, Video Electronics Standards Association (VESA) local buses, and Peripheral Component Interconnect (PCI) buses.

[0122] Electronic devices typically include a variety of computer-readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, and removable and non-removable media.

[0123] The memory may include computer-readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic devices may further include other removable / non-removable, volatile / non-volatile computer device storage media. By way of example only, the storage system may be used to read and write non-removable, non-volatile magnetic media.

[0124] The electronic device can also communicate with one or more external devices (e.g., keyboard, pointing device, camera, etc.), may include a display, and may communicate with one or more devices that enable a user to interact with the electronic device, and / or with any device that enables the electronic device to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via an input / output (I / O) interface. Furthermore, the electronic device can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public networks, such as the Internet) via a network adapter. The network adapter communicates with other modules of the electronic device via a bus. The processor executes various functional applications and data processing by running programs stored in memory, such as implementing the interactive patrol robot intelligent agent system based on a large language model provided in the above embodiments of the present invention.

[0125] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An interactive patrol robot intelligent agent system based on a large language model, characterized in that, include: The system comprises a terminal interaction layer, a cloud decision-making layer, and a robot execution layer. The terminal interaction layer communicates with the cloud decision-making layer via a wireless network, and the cloud decision-making layer communicates with the robot execution layer via a wireless network. The terminal interaction layer is used to receive natural language commands or shortcut commands input by the user and send them to the cloud decision layer. It is also used to receive and display the task execution status and result information returned by the cloud decision layer. The cloud-based decision-making layer is deployed on a cloud server and is used to receive user instruction information from the terminal interaction layer, robot status information and task execution result information from the robot execution layer, as well as pre-stored patrol scene map information. It performs semantic understanding and logical reasoning on multi-source heterogeneous data through a large language model, converts the user's natural language instructions into standardized instruction data packets that can be executed by the robot, and sends them to the robot execution layer. The robot execution layer is the autonomous mobile robot body, which is used to receive standardized instruction data packets issued by the cloud decision layer, parse them, execute corresponding patrol tasks, and report the robot's own status information, environmental data collected by sensors, and analysis results to the cloud decision layer in real time.

2. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The terminal interaction layer includes a dialogue interaction module and a shortcut command module. The dialogue interaction module receives text or voice commands input by the user, converts the voice commands into text commands, and sends them to the cloud decision-making layer. It also receives feedback information in text or voice form returned by the cloud decision-making layer and presents it to the user through a display screen and a speaker. The shortcut command module contains multiple graphical touch buttons, each associated with a preset patrol task shortcut command. When the user clicks a graphical touch button, the shortcut command module automatically generates a corresponding standardized shortcut command data packet and sends it to the cloud decision-making layer.

3. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The cloud-based decision-making layer includes a data receiving interface, a large language model processing unit, and an instruction structured output unit. The data receiving interface is communicatively connected to the terminal interaction layer and the robot execution layer, respectively, and is used to receive user instruction information from the terminal interaction layer, robot local status information from the robot execution layer, current task execution status information, environmental data collected by sensors, and AI analysis results. It is also used to receive pre-stored patrol scene map information. The large language model processing unit is loaded with a pre-trained large language model. Its input end is connected to the data receiving interface. It is used to fuse and process the received multi-source heterogeneous data, perform semantic understanding of the user's natural language commands, and perform logical reasoning by combining patrol scene map information and robot current state information to decompose the task's time requirements, location requirements, action requirements, functional requirements, and feedback requirements. The input end of the instruction structured output unit is connected to the output end of the large language model processing unit. It is used to convert the reasoning results output by the large language model processing unit into standardized instruction data packets that can be executed by the robot, and send them to the robot execution layer via a wireless network.

4. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The standardized instruction data packet includes at least a time field, a location field, a task field, a function field, and a feedback field; the time field is used to specify the execution time of the task, including three types: immediate execution, scheduled execution, and periodic execution.

5. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The location field specifies the execution location of the task, including the geographic coordinates of the target point, the area range, and path planning requirements. The task field specifies the specific action type that the robot needs to perform, including navigation movement, stationary stop, rotational scanning, following movement, and emergency alarm. The function field specifies the sensors and analysis functions that the robot needs to activate during task execution, including camera video acquisition, LiDAR obstacle avoidance, smoke detection, temperature detection, sound detection, face recognition, and abnormal behavior analysis. The feedback field specifies the data type of results that the robot needs to send back to the cloud decision-making layer, including real-time location information, battery information, speed information, raw sensor data, AI analysis results, video streams, and images.

6. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The robot's execution layer includes a chassis movement mechanism, a sensor suite, an execution controller, and a status feedback module. The chassis movement mechanism, using a wheeled or tracked chassis, is used to enable the robot to move, turn, and stop according to the instructions of the execution controller. The sensor suite includes at least a camera, lidar, microphone array, smoke sensor, temperature sensor, and infrared sensor, used to collect image data, point cloud data, sound data, smoke concentration data, temperature data, and infrared radiation data in the patrol environment. The execution controller communicates with the cloud-based decision-making layer via a wireless communication module, receives standardized instruction data packets from the cloud-based decision-making layer, parses the instruction data packets, generates corresponding control signals, and controls the chassis movement mechanism and sensor suite to perform corresponding tasks. The status feedback module is connected to the chassis moving mechanism, sensor kit and execution controller respectively, and is used to collect the robot's local status information in real time, including battery level, position, speed, attitude, fault information, as well as environmental data and AI analysis results collected by the sensor kit, and report them to the cloud decision layer in real time via wireless network.

7. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The large language model processing unit also includes an instruction error correction module and a task scheduling module. The instruction error correction module is used to perform grammatical error correction and semantic completion on the natural language instructions input by the user. When the instructions input by the user contain grammatical errors, semantic ambiguity, or incomplete information, it obtains the missing key information through multi-turn dialogue interaction with the user to ensure the accuracy and completeness of the instructions. The task scheduling module is used to reasonably schedule and sort multiple tasks according to the robot's current state, task priority, and urgency. When multiple tasks are issued at the same time, the task with higher urgency is executed first, and the execution time of other tasks is arranged to avoid task conflicts.

8. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The cloud-based decision-making layer also includes a map management module and a data storage module. The map management module is used to store and manage high-precision electronic maps of the patrol scenario, including map creation, updating, deletion, and query functions. It also supports real-time map updates, ensuring the accuracy of robot navigation by updating map information promptly when the patrol scenario changes. The data storage module is used to store user command history, task execution records, sensor data, AI analysis results, and system log data.

9. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The robot execution layer also includes a local emergency processing module, which is connected to the execution controller. When communication between the robot and the cloud decision layer is interrupted, the module automatically switches to local emergency mode and executes preset emergency tasks, including returning to the charging station, standing by in place, and issuing an audible and visual alarm.

10. The interactive patrol robot intelligent agent system based on a large language model according to claim 1, characterized in that, The terminal interaction layer also includes a task monitoring module, which is used to display the robot's current position, task execution status, sensor data, and video stream in real time.

Images