Indoor robot inspection method and device based on large language model

Abstract

The embodiment of the invention discloses an indoor robot inspection method and device based on a large language model. A specific implementation mode of the method comprises the following steps: constructing an occupation grid map of an indoor environment in real time based on sensor data acquired in the advancing process of an indoor robot; generating a collision-free topological graph; dividing the collision-free topological graph into a global belief graph and a local belief graph to construct a hierarchical belief graph; for the hierarchical belief graph, the following control steps are executed: obtaining high-level exploration plan information by using a pre-trained large language model; generating a target node by using a pre-trained strategy model; generating a collision-free motion track between the current position of the indoor robot and the target node based on the collision-free topological graph and the occupied grid map; and controlling the indoor robot to execute inspection operation. According to the embodiment, indoor inspection driven by the natural language instruction can be realized, and the inspection efficiency of the indoor robot is improved.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, and more specifically to an indoor robot inspection method and apparatus based on a large language model. Background Technology

[0002] Currently, in the field of robot inspection, traditional methods rely on fixed path planning algorithms or pre-programmed inspection points.

[0003] However, when using the above method for indoor robot inspection, the following technical problems often arise: Traditional inspection methods often result in short-sighted and redundant inspection paths, leading to low inspection efficiency, increased energy consumption, and excessively long inspection times. Furthermore, they cannot flexibly inject human intent through natural language (e.g., prioritizing inspection of specific areas), making it difficult to adapt to the personalized inspection needs of complex indoor scenarios, increasing the cost of manual intervention, and wasting a significant amount of energy resources.

[0004] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0006] Some embodiments of this disclosure propose indoor robot inspection methods, apparatuses, electronic devices, and computer-readable media based on large language models to solve one or more of the technical problems mentioned in the background section above.

[0007] In a first aspect, some embodiments of this disclosure provide an indoor robot inspection method based on a large language model, comprising: responding to the indoor robot starting an inspection task, constructing an occupied grid map of the indoor environment in real time based on sensor data collected during the indoor robot's movement; generating a collision-free topology map based on the occupied grid map; dividing the collision-free topology map into a global belief map and a local belief map to construct a hierarchical belief map; and performing the following control steps for the hierarchical belief map: obtaining high-level exploration plan information using a pre-trained large language model based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions; generating a target node using a pre-trained policy model based on the local belief map in the hierarchical belief map and the high-level exploration plan information; generating a collision-free motion trajectory between the current position of the indoor robot and the target node based on the collision-free topology map and the occupied grid map; and controlling the indoor robot to perform inspection operations based on the collision-free motion trajectory.

[0008] Secondly, some embodiments of this disclosure provide an indoor robot inspection device based on a large language model, comprising: a construction unit configured to, in response to the indoor robot starting an inspection task, construct an occupied grid map of the indoor environment in real time based on sensor data collected during the movement of the indoor robot; a generation unit configured to generate a collision-free topology map based on the occupied grid map; a partitioning unit configured to partition the collision-free topology map into a global belief map and a local belief map to construct a hierarchical belief map; and a control unit configured to perform the following control steps for the hierarchical belief map: obtaining high-level exploration plan information using a pre-trained large language model based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions; generating a target node using a pre-trained strategy model based on the local belief map in the hierarchical belief map and the high-level exploration plan information; generating a collision-free motion trajectory between the current position of the indoor robot and the target node based on the collision-free topology map and the occupied grid map; and controlling the indoor robot to perform inspection operations based on the collision-free motion trajectory.

[0009] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0010] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0011] The above-described embodiments of this disclosure have the following beneficial effects: the efficiency of indoor robot inspection based on a large language model, as described in some embodiments of this disclosure, is improved. Specifically, the reason for the low efficiency of inspection is that traditional inspection methods rely on fixed path planning algorithms or pre-programmed inspection points, lacking the ability to adapt to dynamic environmental changes and the ability to flexibly adjust based on high-level semantic instructions. Robots cannot understand the real-time intentions of human operators and struggle to intelligently replan based on on-site conditions (e.g., specific area priorities, obstacle changes), leading to path redundancy, repeated coverage, or missed inspections of key areas, thus significantly reducing overall inspection efficiency. Based on this, the indoor robot inspection method based on a large language model, as described in some embodiments of this disclosure, firstly, in response to the indoor robot starting its inspection task, constructs an occupancy grid map of the indoor environment in real time based on sensor data collected during the robot's movement. By constructing the occupancy grid map in real time, an accurate and dynamic environmental model foundation is provided for subsequent path planning and collision detection, ensuring that the robot's perception of the surrounding space is always synchronized with the actual situation. Then, a collision-free topology map is generated based on the occupancy grid map. The continuous occupied grid map is abstracted into a discrete graph structure. Collision-free paths are connected as edges, transforming the complex geometric obstacle avoidance problem into an efficient graph search problem. This significantly reduces the complexity of path planning, providing a lightweight, structured environmental representation for subsequent rapid global planning and local decision-making. Next, the collision-free topology is divided into a global belief graph and a local belief graph to construct a hierarchical belief graph. This hierarchical representation divides the unified collision-free topology into a high-resolution local belief graph (for real-time control) and a sparse global belief graph (for high-level reasoning) based on spatial scale. This hierarchical representation resolves the contradiction between the computational overhead of global planning and the accuracy requirements of local decision-making that a single map cannot easily achieve. Finally, for the hierarchical belief graph, the following control steps are performed: Based on the global belief graph and the acquired natural language preference instructions, a pre-trained large language model is used to obtain high-level exploration plan information. The graph structure of the global belief graph is converted into text and integrated with the natural language preference instructions. The high-level exploration plan information is then generated through the large language model, achieving accurate conversion of human intent into global planning. Based on the local belief graph in the aforementioned hierarchical belief graph and the high-level exploration plan information, a pre-trained policy model is used to generate target nodes. By fusing the local belief graph and the high-level exploration plan information, the policy model filters target nodes, achieving a connection between global intent and local execution, ensuring that the robot's movement direction aligns with preferences and global optimality. Based on the aforementioned collision-free topology graph and occupied grid map, a collision-free trajectory is generated between the current position of the indoor robot and the aforementioned target nodes. Combining the shortest path generated from the collision-free topology graph with collision detection from the occupied grid map, a smooth and feasible trajectory is generated, balancing path efficiency and collision-free safety, providing an execution basis for precise robot movement.Based on the aforementioned collision-free motion trajectory, the indoor robot is controlled to perform inspection operations. The collision-free motion trajectory is converted into control commands, driving the robot's physical movement, and simultaneously collecting new data to update the occupied grid map, forming a closed loop of "planning -> execution -> perception". This allows the robot to adapt to environmental changes and autonomously advance its tasks, achieving autonomous inspection. Attached Figure Description

[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0013] Figure 1 This is a flowchart of some embodiments of the indoor robot inspection method based on a large language model according to the present disclosure; Figure 2 This is a structural schematic diagram of some embodiments of the indoor robot inspection device based on a large language model according to the present disclosure; Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0016] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0017] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0018] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0019] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] refer to Figure 1 The diagram illustrates a flow 100 of some embodiments of the indoor robot inspection method based on a large language model according to the present disclosure. This indoor robot inspection method based on a large language model includes the following steps: Step 101: In response to the indoor robot starting its inspection task, a grid map of the indoor environment is constructed in real time based on the sensor data collected during the robot's movement.

[0021] In some embodiments, the executor (e.g., an electronic device) of the above-described indoor robot inspection method based on a large language model can be either hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster consisting of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software programs or software modules to provide distributed services, or as a single software program or software module. No specific limitations are made here.

[0022] In other embodiments, the aforementioned execution entity can, in response to the indoor robot initiating an inspection task, construct a real-time occupancy grid map of the indoor environment based on sensor data collected during the indoor robot's movement. The indoor robot can be an intelligent machine device that performs autonomous movement and inspection within an indoor environment, such as a wheeled robot used for inspection. The inspection task can be an environmental detection and status check performed by the indoor robot on a designated area; for example, the inspection task may include: equipment inspection in an office building corridor or security inspection in an exhibition hall. The indoor environment can be a closed or semi-closed indoor space covered by the indoor robot's inspection, such as an office building floor, a shopping mall exhibition hall, or a factory workshop. The sensor data can be raw environmental information collected by the indoor robot through sensors. For example, the sensor data may include: point cloud data obtained from LiDAR scanning or depth images acquired by a depth camera. The occupancy grid map can be a map that divides the indoor environment into uniform grids and marks the state of each grid (e.g., free, occupied, unknown). For example, the occupancy grid map can be an occupancy grid map with a resolution of 5 cm, marking walls (occupied) and passable areas (free).

[0023] In some optional implementations of certain embodiments, the aforementioned execution entity may, in response to the indoor robot initiating an inspection task, construct a real-time occupancy grid map of the indoor environment based on sensor data collected during the indoor robot's movement, which may include the following steps: The first step is to transform the sensor data and the current pose data of the indoor robot into standardized observation data in a world coordinate system. The current pose data can be the robot's real-time position and attitude information in the world coordinate system. For example, the current pose data could be: x=5m, y=3m, heading angle=90°. The standardized observation data can be a normalized representation of the sensor data after transformation to a unified world coordinate system. In practice, firstly, a coordinate transformation matrix is ​​used to transform each point in the sensor data from the sensor coordinate system to the world coordinate system to generate standardized observation data.

[0024] The second step involves updating the pre-stored occupancy grid map based on the standardized observation data, resulting in an occupancy grid map. This pre-stored occupancy grid map can be a previously constructed or initialized grid map, serving as the basis for the update. For example, it could be a map saved from the last inspection and loaded at the start of the task, or an initially completely unknown map. In practice, firstly, each point in the standardized observation data is mapped to a grid in the pre-stored occupancy grid map. Then, the occupancy probability of the corresponding grid and its adjacent grids is updated based on whether the point represents an obstacle (e.g., a laser point hitting an object's surface). (For example, an anti-sensor model can be used). Finally, the update is completed by traversing all observation points, resulting in an occupancy grid map reflecting the latest environmental understanding.

[0025] Step 102: Generate a collision-free topology map based on the occupied grid map.

[0026] In some embodiments, the aforementioned execution entity can generate a collision-free topology graph based on an occupied grid map. This collision-free topology graph can be a graph data structure, where nodes represent feasible locations and edges represent collision-free feasible paths between nodes. For example, the collision-free topology graph could be a network graph using nodes and edges to represent various safety passages within a room.

[0027] In some optional implementations of certain embodiments, the aforementioned execution entity may generate a collision-free topology map based on an occupied grid map, which may include the following steps: The first step is to extract walkable areas from the occupied grid map. These walkable areas can be sets of grid cells marked "free" in the grid map, representing physical spaces that the robot can safely traverse. For example, a walkable area could be a floor area in an office that is not occupied by desks and chairs. In practice, first, each grid cell in the occupied grid map is traversed. Then, the grid cells marked "free" are extracted to form a binary mask or set of coordinates, which serves as the walkable area.

[0028] The second step involves uniformly sampling within the accessible area to generate a candidate viewpoint set. This candidate viewpoint set can be a series of coordinate points uniformly sampled within the accessible area. In practice, firstly, within the accessible area (e.g., a continuous set of grids), sampling point coordinates are generated at a set sampling interval (e.g., every 0.5 meters). Then, these sampling points are collected to form the candidate viewpoint set.

[0029] The third step is to determine the nearest neighbor nodes of the target number for each candidate viewpoint in the aforementioned candidate viewpoint set to generate an initial topology graph. The target number can be a preset number of nearest neighbor nodes to be connected for each candidate viewpoint. The nearest neighbor nodes can be several other points in the candidate viewpoint set that are closest to the candidate viewpoint in terms of Euclidean distance. The initial topology graph can be an initial graph structure formed by connecting each candidate viewpoint with its nearest neighbor nodes; edges may cross obstacles. In practice, firstly, for each candidate viewpoint in the candidate viewpoint set, the Euclidean distance from that candidate viewpoint to all other candidate viewpoints in the set is calculated. Then, for each point, the k nearest points (the target number) are selected as its neighbors. Finally, each point is connected to its k neighbors with edges to form the initial topology graph.

[0030] The fourth step involves performing collision detection on each candidate edge in the initial topology graph to generate a first collision detection result. This first collision detection can involve checking whether an edge (line segment) intersects with an obstacle region in the grid map. The first collision detection result can be a Boolean value (safe / collision) obtained after performing collision detection on each candidate edge. In practice, firstly, each candidate edge in the initial topology graph is obtained, defined by two viewpoint coordinates as a line segment. Then, dense sampling (six days, one point every 0.05 meters) is performed on this line segment, checking whether the grid corresponding to each sampling point is an "obstacle" in the map. Finally, if all sampling points correspond to "empty" grids, the edge is considered safe; otherwise, it is considered a collision.

[0031] Fifth, based on the first collision detection results, the candidate edges of the initial topology graph are filtered to obtain a filtered candidate edge set. The candidate edges can be edges connecting two candidate viewpoints in the initial topology graph. The filtered candidate edge set can be the set of safe candidate edges retained after collision detection. In practice, first, the first collision detection results are traversed. Then, each candidate edge marked as "collision" is removed from the edge set of the initial topology graph. Finally, the remaining edges constitute the filtered candidate edge set.

[0032] Step 6: Based on the candidate viewpoint set and the filtered candidate edge set described above, generate a collision-free topology graph. In practice, firstly, the candidate viewpoint set is treated as a node set. Then, the filtered candidate edge set is treated as an edge set. Finally, the node set and edge set are combined to obtain the collision-free topology graph.

[0033] Step 103: Divide the collision-free topology graph into a global belief graph and a local belief graph to construct a hierarchical belief graph.

[0034] In some embodiments, the aforementioned execution entity can divide the collision-free topology graph into a global belief graph and a local belief graph to construct a hierarchical belief graph. In practice, the global belief graph can be a high-level abstract graph obtained by dividing the collision-free topology graph into communities, where nodes represent regions (communities) and edges represent connectivity between regions. For example, the global belief graph can be a network composed of nodes divided from an office building topology graph (e.g., nodes could be "lobby," "Area A corridor," or "Area B office area"). The local belief graph can be a local high-resolution graph extracted from the collision-free topology graph, centered on the current position of the indoor robot, for real-time decision-making. For example, the local belief graph can be a network composed of path nodes and connecting edges within a 10-meter radius around the robot. The hierarchical belief graph can be a two-layer environment representation model composed of a global belief graph (macro-region network) and a local belief graph (micro-path network).

[0035] In addressing the technical challenges mentioned above, the application scenario—large-scale, structurally complex indoor inspection scenarios requiring long-term autonomous operation, such as large general hospitals, smart logistics warehouses, or multi-story parking garages—often presents the following challenges: a single-scale environmental representation cannot simultaneously support efficient global task allocation and precise local obstacle avoidance decision-making; real-time local decision-making is prone to getting stuck in local optima, resulting in lengthy overall paths, low exploration efficiency, and wasted inspection time. Considering the following requirements for this application scenario—scalability in ultra-large area environments, high-precision obstacle avoidance capabilities in complex local scenarios, and real-time coordination between global targets and local actions—we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity may divide the collision-free topology graph into a global belief graph and a local belief graph to construct a hierarchical belief graph, which may include the following steps: The first step is to determine a local sliding window based on the current position of the indoor robot. The current position can be the real-time coordinates of the indoor robot in the world coordinate system (e.g., x=12.5m, y=8.2m). The local sliding window can be a square area centered on the current position. In practice, first, the current position coordinates of the indoor robot are obtained. Then, a square area of ​​a preset size (e.g., 20m × 20m) is drawn on the environmental map centered on this point. Finally, this area is the local sliding window. For example, if the current position is (10,10), the window range is from (0,0) to (20,20).

[0036] The second step is to filter out the nodes located within the local sliding window from the collision-free topology graph to generate a local node set. This local node set can be any collision-free topology graph node that falls within the local sliding window. For example, the coordinates of 15 path points contained within the window. In practice, first, the coordinates of each node in the collision-free topology graph are traversed. Then, it is determined whether each node is within the local sliding window. Finally, all nodes falling within the local sliding window are collected to form the local node set.

[0037] The third step is to extract all edges connecting any two nodes in the local node set from the collision-free topology graph, generating a local edge set. This local edge set consists of edges connecting any two nodes in the local node set that exist in the collision-free topology graph. In practice, first, all edges in the collision-free topology graph are traversed. Then, it is checked whether the two endpoints of each edge belong to the local node set. Finally, the edges that meet the conditions are extracted to form the local edge set.

[0038] The fourth step is to construct the local belief graph based on the aforementioned set of local edges and local nodes. In practice, firstly, the set of local nodes is used as the node set of the graph. Then, the set of local edges is used as the edge set of the graph. Finally, combining the two constructs the local belief graph.

[0039] The fifth step involves generating a global community set based on the aforementioned collision-free topology graph using a community detection algorithm. This global community set can be a set of node clusters obtained by partitioning the complete collision-free topology graph using a community detection algorithm (e.g., Louvain's algorithm). For example, 200 nodes can be clustered into 8 communities, with each community representing a densely connected region.

[0040] The sixth step is to map each community in the aforementioned global community set to a global node, generating a global node set. In practice, first, each community in the global community set is traversed. Then, a new, unique identifier is created for each community as a global node. Finally, the identifiers of all communities are combined into a global node set. For example, seven communities are mapped to global nodes G1 to G7.

[0041] Step 7: Based on the cross-community edges in the collision-free topology graph described above, generate a global edge set. In practice, first, traverse all edges in the collision-free topology graph. Then, check whether the two endpoints of each edge belong to different communities in the global community set. Finally, if they belong to different communities, establish a global edge between the global nodes corresponding to these two communities to generate the global edge set.

[0042] Step 8: Based on the aforementioned global node set and global edge set, construct the global belief graph. In practice, first, the global node set is used as the node set. Then, the global edge set is used as the edge set. Finally, combining the two constructs the global belief graph.

[0043] Step nine involves constructing a hierarchical belief graph based on the global and local belief graphs described above. In practice, the local belief graph is first used as the bottom layer of the hierarchical model. Then, the global belief graph is used as the top layer. Finally, the two graphs are linked through data associations (e.g., which global community a local node belongs to) to form a unified hierarchical belief graph.

[0044] The above-described operational steps, combined with step 1044, constitute an inventive point of this disclosure, solving the technical problem mentioned in the background art: "A single-scale environmental representation cannot simultaneously support efficient global task allocation and refined local obstacle avoidance decision-making, and real-time local decisions are prone to getting trapped in local optima, resulting in a lengthy overall path, low exploration efficiency, and wasted inspection time." The reasons for these technical problems are as follows: Traditional methods either use dense maps for locally accurate but globally inefficient searches, or use abstract topologies for globally fast but locally responsive planning; the two are disconnected and cannot coordinate. This invention constructs a local belief graph for millisecond-level real-time decision-making, while simultaneously constructing a global belief graph for second-level macro-level reasoning, and establishes a hierarchical relationship between the two, achieving efficient collaborative operation between high-level strategic planning and low-level real-time control, saving path redundancy and time costs caused by global planning computation delays and local decision-making errors.

[0045] Step 104, for the hierarchical belief map, perform the following control steps: Step 1041: Based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions, a pre-trained large language model is used to obtain high-level exploration plan information. The aforementioned natural language preference instructions can be exploration task requirements or constraints expressed by the user in natural language. For example, the natural language preference instruction could be "Please explore the meeting room on the east side first, avoiding densely populated areas." The aforementioned pre-trained large language model can be a deep learning model (e.g., an LLaMA model) trained on massive amounts of text data and possessing natural language understanding and reasoning capabilities. It combines the global belief map with the natural language preference instructions to generate logically consistent and constraint-compliant high-level exploration plan information through reasoning. The structure of the aforementioned large language model can be summarized as "input encoding layer (word segmentation + embedding + positional encoding) -> multi-layer Transformer inference layer (self-attention mechanism and feedforward neural network) -> output decoding layer (linear projection, softmax function)," with each layer stacked sequentially to abstract and generate high-level exploration plan information. This high-level exploration plan information can be a planning result generated by the large language model, guiding the macroscopic actions of the indoor robot. For example, the above-mentioned high-level exploration plan information could be "Area access order: [Lobby -> East Corridor -> Meeting Room 101]".

[0046] In some embodiments, the aforementioned executing entity may obtain high-level exploration plan information based on the global belief map in the aforementioned hierarchical belief map and the acquired natural language preference instructions, using a pre-trained large language model.

[0047] In some optional implementations of certain embodiments, the aforementioned execution entity can obtain high-level exploration plan information based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions, using a pre-trained large language model. This may include the following steps: The first step is to convert the nodes, edges, and exploration status information of the global belief graph into structured text descriptions. The exploration status information describes whether each region node in the global belief graph has been explored. For example, the node "Area A" might have a status of "80% explored." This structured text description can be a structured representation of the graph structure information (nodes, edges, and exploration status information) of the global belief graph. In practice, first, all nodes and edges of the global belief graph are traversed, and the exploration status of each node (e.g., "explored percentage") is read. Then, it is organized into a natural language sentence according to a preset template. Finally, a structured text description is generated. For example, "The environment includes the following areas: the lobby (60% explored), the east corridor (unexplored), and conference room 101 (unexplored). The lobby is connected to the east corridor, and the east corridor is connected to conference room 101." The second step involves combining the aforementioned natural language preference instructions, structured text descriptions, and acquired historical exploration information to generate the input information for the large language model, resulting in an input sequence. The input information can be a text to be processed that combines natural language preference instructions, structured text descriptions, and historical exploration information. The historical exploration information can be data recording the areas, paths, and task statuses already explored by the indoor robot. The input sequence can be a numerical sequence acceptable to the large language model, formed after processing the input information such as word segmentation. In practice, first, user instructions (e.g., "prioritize checking the meeting room") and historical information (e.g., "already visited the lobby") are acquired. Then, these texts are sequentially concatenated with the structured description. Finally, the concatenated complete text is used as the input information for the large language model to obtain the input sequence.

[0048] The third step involves iteratively reasoning through the input sequence using the aforementioned large language model to generate the original output sequence. This original output sequence can be the original token sequence generated by the large language model. In practice, the input text is first segmented into words, converting it into a numerical sequence (input sequence) that the model can process. Then, this sequence is input into the pre-trained large language model. Finally, the model performs autoregressive reasoning, generating output token by token to form the original output sequence.

[0049] The fourth step involves decoding the original output sequence to generate an adjudication token and a high-level access sequence. The adjudication token can be a special token output by the large language model, indicating whether it has completed reasoning and is ready to provide an answer. For example, "Answer" or "Unknown". The high-level access sequence can be a sequence of global nodes suggested for access, output by the large language model, such as [G1, G3, G5]. In practice, first, the original output sequence generated by the large language model is parsed. Then, the special adjudication token at the beginning of the sequence (e.g., "Answer" or "Unknown") is identified. Finally, the portion following the adjudication token is extracted as the suggested high-level access sequence.

[0050] Fifth, in response to the aforementioned ruling marker indicating the completion of reasoning, the aforementioned high-level access sequence is extracted to obtain candidate high-level exploration plan information. This candidate high-level exploration plan information may be the high-level planning results obtained from preliminary parsing, without yet undergoing structured mapping. In practice, first, it is checked whether the ruling marker is a marker indicating the completion of reasoning (e.g., "Answer"). Then, if the marker is valid, the extracted high-level access sequence is used as candidate high-level exploration plan information. For example, if the ruling marker is "Answer", then the access sequence [G3, G5, G2] is accepted as a candidate plan.

[0051] Step 6: Map the aforementioned candidate high-rise exploration plan information to high-rise exploration plan information with regional priorities and access orders. The regional priorities and access orders can be specific representations of the planning results, clearly defining the exploration order and importance of each region. For example, {Priority: East Zone (High), Order: [Region A, Region B]}. In practice, firstly, the symbols in the candidate high-rise exploration plan information (e.g., G3) are mapped back to the actual regions they represent (e.g., "Conference Room 101"). Then, the priority and access order of each region are determined according to the sequence. Finally, structured high-rise exploration plan information is generated. For example, [G3, G5, G2] is mapped as: priority order ["Conference Room 101" (High), "Power Distribution Room" (Medium), "West Corridor" (Low)].

[0052] Step 1042: Based on the local belief graph and high-level exploration plan information in the hierarchical belief graph, generate the target node using a pre-trained policy model.

[0053] In some embodiments, the aforementioned execution entity can generate target nodes based on the local belief graph in the hierarchical belief graph and the high-level exploration plan information, using a pre-trained policy model. The pre-trained policy model can be a decision model based on a graph neural network (GNN), with structured input information (including node features and adjacency relationships) as input and outputting the selection probability or priority score for each candidate node, used to decide the robot's next target. The pre-trained policy model may include: an input projection layer, an adjacency-constrained attention layer (core), a feedforward nonlinear layer, a residual connection and normalization layer, and a decoding / decision layer. Data flows in the order of input projection -> attention layer -> feedforward layer -> residual normalization (multiple layers can be stacked) -> decoding layer. The output of each layer serves as the input for the next layer; adjacency constraint information propagates only in the attention layer; and residual connections ensure gradient stability. The target node can be a local belief graph node output by the policy model, indicating the robot's next destination.

[0054] In some optional implementations of certain embodiments, the execution entity may generate target nodes based on the local belief graph in the hierarchical belief graph and the high-level exploration plan information, using a pre-trained policy model, which may include the following steps: The first step is to construct a node feature vector for each node in the local belief graph to generate a node feature vector set. This node feature vector set can be the attribute encoding set for each node in the local belief graph. In practice, first, all nodes in the local belief graph are traversed. Then, for each node, its geometric coordinates, exploration utility value (e.g., the number of observable fronts), and relative position to the robot's current position are calculated. Finally, these are concatenated to form a fixed-dimensional feature vector, and the feature vectors of all nodes constitute the node feature vector set. For example, for node v1, its coordinates (2.1, 4.3) are extracted, and its utility value of 0.75 is calculated, resulting in the feature vector [2.1, 4.3, 0.75].

[0055] The second step involves encoding the aforementioned high-level exploration plan information into guiding features associated with each node in the local belief graph. These guiding features can be constraints that map the high-level exploration plan to each local node. For example, a guiding feature could be a 0 / 1 feature indicating whether a node "belongs to the priority area kitchen". In practice, first, the high-level exploration plan information (e.g., "the next priority area is the kitchen") is parsed. Then, one or more features associated with this plan are calculated for each node in the local belief graph. Finally, a guiding feature vector is generated for each node. For example, for each node, its location within the "kitchen" area is calculated, resulting in a binary feature [0 or 1]; simultaneously, its Euclidean distance to the kitchen entrance is calculated, resulting in a continuous feature.

[0056] The third step involves fusing the aforementioned node feature vector set with the aforementioned guiding feature vector to generate fused node feature information. This fused node feature information can be an enhanced feature resulting from combining the node's own features with the guiding feature. In practice, firstly, the node feature vector set and the guiding feature vector are aligned along the feature dimension. Then, the two feature vectors corresponding to each node are directly concatenated. Finally, a new, higher-dimensional set of fused node feature information is generated. For example, the original feature [2.1, 4.3, 0.75] of node v1 is concatenated with the guiding feature [1, 2.5] to obtain the fused feature [2.1, 4.3, 0.75, 1, 2.5].

[0057] The fourth step involves transforming the fused node feature information and the adjacency relationships of the local belief graph into structured input information. These adjacency relationships can be the connections between nodes in the local belief graph. For example, if node A and node B are connected by an edge, the adjacency relationship contains the edge (A, B), indicating that the robot can move directly from A to B. The structured input information can be complete graph data including the fused node feature information and the adjacency relationships of the local belief graph. In practice, firstly, the fused feature vectors of each node are stacked to form a node feature matrix. Then, the adjacency matrix is ​​extracted from the local belief graph to represent the connections between nodes. Finally, these two matrices are combined into a standard graph data structure as the structured input information.

[0058] The fifth step involves feeding the structured input information into the pre-trained policy model to generate priority information for each candidate node. This priority information can be a "value" or "selection probability" score determined for each node. In practice, first, the structured input information generated in the fourth step is fed into the pre-trained policy model. Then, the policy model propagates and aggregates features through its internal graph neural network layers (e.g., graph attention layers). Finally, the model's output layer calculates a scalar score for each node to generate priority information.

[0059] The sixth step is to determine the target node based on the priority information of each candidate node. In practice, firstly, the node index with the highest score in the priority information vector is selected. Finally, the local belief graph node corresponding to that index is determined as the target node. For example, if the priority vector is [0.2, 0.1, 0.6, 0.05], and the maximum value of 0.6 corresponds to index 2, then the third node in the local graph (index starts from 0) is selected as the target node.

[0060] Step 1043: Based on the collision-free topology map and the occupied grid map, generate the collision-free motion trajectory between the current position of the indoor robot and the target node.

[0061] In some embodiments, the aforementioned execution entity can generate a collision-free motion trajectory between the current position of the indoor robot and the target node based on the aforementioned collision-free topology map and occupied grid map. The collision-free motion trajectory can be a continuous sequence of positions (or postures) that the indoor robot can safely execute over time. For example, the collision-free motion trajectory can be a series of coordinate points from point (1,1) to point (5,5), ensuring that it does not collide with any obstacles.

[0062] In addressing the technical challenges mentioned above, the application scenario—large, complex, and partially unknown indoor inspection scenarios such as large warehouses, data centers, or hospitals—often presents the following challenges: paths planned on sparse topologies may collide with minor obstacles in continuous physical space; dynamic environmental changes (e.g., the appearance of temporary obstacles) may cause pre-defined paths to fail, wasting computational, time, and energy resources, leading to task interruptions, robot malfunctions, or safety risks. Considering the following requirements for this application scenario: paths must be absolutely safe and collision-free; planning must balance efficiency and accuracy; and the system must be able to cope with environmental uncertainties, we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity can generate a collision-free motion trajectory between the current position of the indoor robot and the target node based on the collision-free topology map and the occupied grid map, which may include the following steps: The first step is to determine the node corresponding to the current position of the indoor robot in the collision-free topology graph, which will serve as the starting node. This starting node can be the graph node in the collision-free topology graph that is closest to the robot's current position. In practice, first, the precise current position coordinates of the indoor robot are obtained. Then, the Euclidean distances from the current position coordinates to each node in the collision-free topology graph are calculated. Finally, the node with the closest distance is selected as the starting node.

[0063] The second step, based on the collision-free topology graph described above, is to determine the shortest path between the starting node and the target node, thereby generating a shortest path node sequence. The shortest path can be a sequence of nodes (or edges) on the collision-free topology graph that minimize the cost from the starting node to the target node. This shortest path node sequence can be an ordered list of nodes constituting the shortest path. In practice, first, a graph is constructed using the edge lengths of the collision-free topology graph as weights. Then, a graph search algorithm (e.g., A or Dijkstra's algorithm) is used to search from the starting node to the target node. Finally, the node sequence returned by the algorithm is extracted as the shortest path node sequence.

[0064] The third step involves connecting adjacent nodes in the shortest path node sequence to generate an initial continuous trajectory. This initial continuous trajectory can be a continuous spatial path formed by connecting the node sequence with straight line segments or curves. In practice, first, the coordinates of each node in the shortest path node sequence are obtained. Then, the coordinates of adjacent nodes are connected sequentially with straight line segments. Finally, all straight line segments are pieced together in order to form a polygonal initial continuous trajectory.

[0065] The fourth step involves performing collision detection on the initial continuous trajectory using the aforementioned occupied grid map to generate a second collision detection result. This second collision detection result can be a Boolean judgment determining whether the initial continuous trajectory intersects with an obstacle in the occupied grid map. For example, if the trajectory intersects with an unmarked obstacle grid on the line segment connecting node v10 to node v20, the result is "collision". In practice, firstly, detection points are sampled on the initial continuous trajectory at high resolution (e.g., 0.01 meters). Then, the state (free / occupied / unknown) of the corresponding grid in the occupied grid map for each detection point is queried. Finally, if any detection point corresponds to an "occupied" state, a collision is determined, and a collision detection result is generated.

[0066] Fifth, in response to the collision detection result mentioned above, the initial continuous trajectory is replanned based on the occupied grid map to generate a replanned trajectory. This replanned trajectory can be a safe trajectory replanned based on more refined map information after the initial trajectory has collided. In practice, firstly, when the collision detection result is "collision," the area near the collision point is marked as a temporary obstacle on the occupied grid map. Then, on the updated occupied grid map, a refined planner (e.g., gradient-based optimization or direct sampling in free space) is used to replan a path around the obstacle. Finally, the new path is converted into the replanned trajectory.

[0067] The sixth step involves smoothing and time-parameterizing the reprogrammed trajectory to generate a collision-free motion trajectory. In practice, firstly, curve fitting (e.g., B-spline) is performed on the trajectory obtained through detection or reprogramming to eliminate sharp corners. Then, a timestamp is assigned to each point on the trajectory based on the maximum speed and acceleration limits of the indoor robot. Finally, a collision-free motion trajectory including position, velocity, and acceleration information is generated.

[0068] The above-described steps, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "Paths planned on sparse topological graphs may collide with minute obstacles in continuous physical space; dynamic environmental changes (e.g., the appearance of temporary obstacles) may cause preset paths to fail, wasting computational, time, and energy resources, leading to task interruption, robot malfunction, or safety risks." The reasons for these technical problems are as follows: planning based on abstract topological graphs ignores the geometric details of continuous space; traditional single-planning lacks feedback and correction mechanisms for environmental changes during execution. This invention, by introducing a collision detection and replanning closed loop based on a precisely occupied grid map on topological planning, achieves dual protection from abstract connectivity to physical safety, and can dynamically adapt to environmental changes, saving the huge time and maintenance costs incurred due to task failure, robot jamming, or the need for manual intervention caused by path collisions or environmental changes.

[0069] Step 1044: Based on the collision-free motion trajectory, control the indoor robot to perform inspection operations.

[0070] In some embodiments, the aforementioned execution entity can control the indoor robot to perform inspection operations based on the aforementioned collision-free motion trajectory. The inspection operation can be the process of the indoor robot moving along a planned path and performing inspections.

[0071] In addressing the technical challenges mentioned above, the application scenario—highly dynamic, high-precision industrial inspection scenarios with partially unknown environments, such as semiconductor workshops, automated warehouses, or nuclear power plant inspections—often presents the following technical problems: static trajectory planning cannot adapt to dynamic disturbances during robot execution (e.g., slippage, load changes); real-time environmental changes (e.g., temporary obstacles, equipment relocation) cause the original trajectory to fail, leading to collisions or task interruptions, wasting valuable operation time and computing resources, resulting in low inspection efficiency and potential production safety risks. Considering the following requirements for this application scenario: millimeter-level trajectory tracking accuracy, millisecond-level dynamic response capability, online environmental awareness and adaptability, and operational safety, we have decided to adopt the following solution: In some optional implementations of certain embodiments, the execution entity can control the indoor robot to perform inspection operations based on the collision-free motion trajectory, which may include the following steps: The first step is to generate a pose and time series and a velocity constraint parameter set based on the aforementioned collision-free motion trajectory. The pose and time series can be a sequence of robot positions and orientations with timestamps. The velocity constraint parameter set can be a set of velocity and acceleration limits for each axis of the robot's motion. For example, a maximum linear velocity of 0.6 m / s and a maximum angular velocity of 1.2 rad / s. In practice, firstly, the path points of the collision-free motion trajectory are analyzed. Then, according to the preset maximum velocity and acceleration constraints, a timestamp and a desired velocity are assigned to each point. Finally, a sequence containing time, position, orientation, and velocity is generated as the pose and time series and velocity constraint parameter set.

[0072] The second step is to convert the aforementioned pose and time sequence into a control instruction set recognizable by the motion controller of the indoor robot. This control instruction set can be a set of signals that directly drive the robot's underlying actuators (e.g., motors). In practice, firstly, the pose difference and time difference between adjacent points in the pose and time sequence are read. Then, the robot's kinematic model is driven based on the pose and time differences to determine the target velocities of the left and right wheels. Finally, the target velocities of the left and right wheels are encoded into a controller protocol in time sequence to form the control instruction set.

[0073] The third step involves performing the following inspection steps at the control command level: Sub-step one involves controlling the motion controller to perform the inspection process based on the aforementioned control instruction set. In practice, firstly, the instructions for the current moment are read from the control instruction set in chronological order. Then, the instructions are sent to the motion controller via a communication interface (e.g., CAN bus). Finally, the controller drives the motors to execute the commands, and the robot begins to move.

[0074] Sub-step two involves acquiring real-time pose feedback data and sensor observation data during the aforementioned inspection process, as well as updating the occupied grid map. The real-time pose feedback data can be the robot's actual pose reported in real-time by sensors (e.g., odometry, IMU). In practice, firstly, the robot's displacement and rotation angles are read in real-time by the encoder and IMU, and fused to obtain the real-time pose feedback data. Simultaneously, the LiDAR continuously scans the environment to obtain point cloud data. Then, the new point cloud data is combined with the current pose to incrementally update the occupied grid map.

[0075] Sub-step three involves determining the trajectory deviation value based on the aforementioned real-time pose feedback data. The sensor observation data can be environmental perception data acquired in real-time during execution. The trajectory deviation value can be the difference between the real-time pose feedback data and the desired pose. In practice, first, the real-time pose feedback data is acquired. Then, the desired pose at the same moment is retrieved from the pose and time series. Finally, the differences between the two in position (Euclidean distance) and orientation (angular difference) are calculated as the trajectory deviation value.

[0076] Sub-step four involves dynamically adjusting the control command set based on the aforementioned trajectory deviation value and speed constraint parameters to generate an adjusted control command set. This adjusted control command set can be a corrected command dynamically determined based on the trajectory deviation value. In practice, first, based on the trajectory deviation value (e.g., 0.5° to the right), the correction direction is determined (a slight left turn adjustment is required). Then, within the speed constraint parameter range (e.g., angular velocity not exceeding 1.0 rad / s), the adjusted left and right wheel speed difference is determined. Finally, a new set of speed commands is generated as the adjusted control command set.

[0077] Sub-step five involves controlling the robot to perform inspections according to the adjusted control instruction set described above. In practice, firstly, the adjusted control instruction set is immediately sent to the motion controller. Then, the controller executes the new instructions, driving the robot to move. Finally, the robot continues its inspection along the desired trajectory according to the corrected instructions.

[0078] Sub-step six: In response to the indoor robot reaching the target node, the inspection operation is completed. In practice, firstly, the robot's real-time pose is continuously compared with the coordinates and orientation of the target node. Then, when the position distance is less than a threshold (e.g., 0.05m) and the orientation error is less than a threshold (e.g., 1°), arrival is determined. Finally, a stop motion command is issued, and the inspection operation is completed.

[0079] Fourth, in response to the indoor robot failing to reach the target node, the adjusted control instruction set is used as the control instruction level, and the inspection steps continue to be executed. In practice, firstly, in response to the indoor robot failing to reach the target node, the adjusted control instruction set is used as the base instruction for the next control cycle. Then, based on this new instruction set, the complete inspection loop is re-executed from the beginning of the sub-step. For example, if the robot deviates from the path due to a temporary obstacle while moving towards the target point, and after adjustment is still 0.2 meters away from the target, the current adjustment instruction is used as the new starting point, and the perception, deviation calculation, and instruction adjustment are continuously executed in a loop until the target node is finally reached. The above-described operation steps, as an inventive point of this disclosure, solve the technical problem mentioned in the background art: "Static trajectory planning cannot adapt to dynamic disturbances during robot execution (e.g., slippage, load changes); real-time environmental changes (e.g., temporary obstacles, equipment relocation) cause the original trajectory to fail, leading to collisions or task interruptions, wasting valuable operation time and computing resources, resulting in low inspection efficiency and easily causing production safety risks." The reasons for the above technical problems are as follows: traditional open-loop control lacks a real-time feedback and compensation mechanism for execution deviations and environmental changes; the planning and control modules are separated, failing to form a closed loop of perception -> decision -> execution. The inventive point of this invention introduces a closed-loop control and dynamic adjustment mechanism based on real-time pose feedback and sensor observation at the trajectory execution layer, achieving high-precision trajectory tracking and real-time compensation for dynamic environmental and self-state disturbances, ensuring safe and continuous task execution, and saving the computing power for replanning, task interruption time, and maintenance and safety costs caused by potential collision accidents due to trajectory tracking failures.

[0080] In some optional implementations of certain embodiments, the aforementioned execution entity may perform the following steps: The first step involves generating an updated hierarchical belief map based on the new sensor data collected during the inspection process, in response to the occupancy grid map meeting preset replanning trigger conditions. These replanning trigger conditions can be a set of preset rules or thresholds for determining whether global replanning is necessary. For example, an increase of more than 10% in the number of nodes in the collision-free topology map, or a 5% increase in the coverage of explored areas. The new sensor data can be the latest sensor data collected during the indoor robot's inspection. The updated hierarchical belief map can be a reconstructed hierarchical belief map based on the latest environmental perception (the updated occupancy grid map). In practice, firstly, during the inspection process, changes in the occupancy grid map are continuously monitored and compared with preset replanning trigger conditions (e.g., graph structure changes exceeding a threshold). Then, once the conditions are met (e.g., the addition of a passable area leading to a 15% increase in the number of nodes in the topology map), the entire process from map updating to hierarchical belief map construction is re-executed based on the latest sensor data. Finally, an updated hierarchical belief map reflecting the latest environment is generated.

[0081] The second step involves, in response to the updated hierarchical belief graph meeting preset conditions, using this updated hierarchical belief graph as the current hierarchical belief graph and continuing the aforementioned control steps. These preset conditions may include a significant change in the structure of the updated hierarchical belief graph (or the receipt of new natural language preference instructions). In practice, first, the updated hierarchical belief graph is checked to determine if it meets the preset conditions. Then, if the conditions are met, this updated hierarchical belief graph is set as the currently used hierarchical belief graph. Finally, based on the updated hierarchical belief graph, the "control steps" are restarted, i.e., the large language model is used to re-plan and generate new target nodes and control the execution of the inspection operation.

[0082] The above-described embodiments of this disclosure have the following beneficial effects: the efficiency of indoor robot inspection based on a large language model, as described in some embodiments of this disclosure, is improved. Specifically, the reason for the low efficiency of inspection is that traditional inspection methods rely on fixed path planning algorithms or pre-programmed inspection points, lacking the ability to adapt to dynamic environmental changes and the ability to flexibly adjust based on high-level semantic instructions. Robots cannot understand the real-time intentions of human operators and struggle to intelligently replan based on on-site conditions (e.g., priority of specific areas, changes in obstacles), leading to path redundancy, repeated coverage, or missed inspections of key areas, thus significantly reducing overall inspection efficiency. Based on this, the indoor robot inspection method based on a large language model, as described in some embodiments of this disclosure, firstly, in response to the indoor robot starting its inspection task, constructs an occupancy grid map of the indoor environment in real time based on sensor data collected during the robot's movement. By constructing the occupancy grid map in real time, an accurate and dynamic environmental model foundation is provided for subsequent path planning and collision detection, ensuring that the robot's perception of the surrounding space is always synchronized with the actual situation. Then, a collision-free topology map is generated based on the occupancy grid map. The continuous occupied grid map is abstracted into a discrete graph structure. Collision-free paths are connected as edges, transforming the complex geometric obstacle avoidance problem into an efficient graph search problem. This significantly reduces the complexity of path planning, providing a lightweight, structured environmental representation for subsequent rapid global planning and local decision-making. Next, the collision-free topology graph is divided into a global belief graph and a local belief graph to construct a hierarchical belief graph. This hierarchical representation divides the unified topology graph into a high-resolution local belief graph (for real-time control) and a sparse global belief graph (for high-level reasoning) based on spatial scale. This hierarchical representation resolves the contradiction between the computational overhead of global planning and the accuracy requirements of local decision-making that a single map cannot easily achieve. Finally, for the hierarchical belief graph, the following control steps are performed: Based on the global belief graph in the hierarchical belief graph and the acquired natural language preference instructions, a pre-trained large language model is used to obtain high-level exploration plan information. The graph structure of the global belief graph is converted into text and integrated with the natural language preference instructions. The high-level exploration plan information is then generated through the large language model, achieving accurate conversion of human intent into global planning. Based on the local belief graph in the aforementioned hierarchical belief graph and the high-level exploration plan information, a pre-trained policy model is used to generate target nodes. By fusing the local belief graph and the high-level exploration plan information, the policy model filters target nodes, achieving a connection between global intent and local execution, ensuring that the robot's movement direction aligns with preferences and global optimality. Based on the aforementioned collision-free topology graph and occupied grid map, a collision-free trajectory is generated between the current position of the indoor robot and the aforementioned target nodes. Combining the shortest path generated from the collision-free topology graph with collision detection from the occupied grid map, a smooth and feasible trajectory is generated, balancing path efficiency and collision-free safety, providing an execution basis for precise robot movement.Based on the aforementioned collision-free motion trajectory, the indoor robot is controlled to perform inspection operations. The collision-free motion trajectory is converted into control commands, driving the robot's physical movement, and simultaneously collecting new data to update the occupied grid map, forming a closed loop of "planning -> execution -> perception". This allows the robot to adapt to environmental changes and autonomously advance its tasks, achieving autonomous inspection.

[0083] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of an indoor robot inspection device based on a large language model. These device embodiments are similar to... Figure 1 Corresponding to the method embodiments shown, this indoor robot inspection device based on a large language model can be specifically applied to various electronic devices.

[0084] like Figure 2 As shown, an indoor robot inspection device 200 based on a large language model includes: a construction unit 201, a generation unit 202, a partitioning unit 203, and a control unit 204. The construction unit 201 is configured to: in response to the indoor robot starting an inspection task, construct an occupied grid map of the indoor environment in real time based on sensor data collected during the robot's movement. The generation unit 202 is configured to: generate a collision-free topology map based on the occupied grid map. The partitioning unit 203 is configured to: partition the collision-free topology map into a global belief map and a local belief map to construct a hierarchical belief map. The control unit 204 is configured to: for the hierarchical belief map, execute the following control steps: based on the global belief map and acquired natural language preference instructions in the hierarchical belief map, obtain high-level exploration plan information using a pre-trained large language model. Based on the local belief map and the high-level exploration plan information in the hierarchical belief map, generate a target node using a pre-trained policy model. Based on the collision-free topology map and the occupied grid map, generate a collision-free motion trajectory between the current position of the indoor robot and the target node. Based on the aforementioned collision-free motion trajectory, the indoor robot is controlled to perform inspection operations.

[0085] It is understandable that the units described in the indoor robot inspection device 200 based on a large language model are related to the reference... Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method are also applicable to the indoor robot inspection 200 with a large language model and the units contained therein, and will not be repeated here.

[0086] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0087] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0088] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0089] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0090] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0091] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0092] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: respond to the indoor robot starting an inspection task; construct an occupied grid map of the indoor environment in real time based on sensor data collected during the indoor robot's movement; generate a collision-free topology map based on the occupied grid map; divide the collision-free topology map into a global belief map and a local belief map to construct a hierarchical belief map; and execute the following control steps for the hierarchical belief map: obtain high-level exploration plan information using a pre-trained large language model based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions; generate a target node using a pre-trained policy model based on the local belief map in the hierarchical belief map and the high-level exploration plan information; generate a collision-free motion trajectory between the current position of the indoor robot and the target node based on the collision-free topology map and the occupied grid map; and control the indoor robot to perform inspection operations based on the collision-free motion trajectory.

[0093] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0095] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a building unit, a generating unit, a partitioning unit, and a control unit. The names of these units do not necessarily limit the specific unit itself. For example, a building unit may be described as "a unit that, in response to an indoor robot starting an inspection task, constructs a real-time occupancy grid map of the indoor environment based on sensor data collected during the robot's movement; and generates a collision-free topology map based on the occupancy grid map."

[0096] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0097] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. An indoor robot inspection method based on a large language model, comprising: In response to the indoor robot starting its inspection task, an indoor environment occupancy grid map is constructed in real time based on sensor data collected during the robot's movement. Generate a collision-free topology map based on the occupied grid map; The collision-free topology graph is divided into a global belief graph and a local belief graph to construct a hierarchical belief graph; For the hierarchical belief map, perform the following control steps: Based on the global belief graph in the hierarchical belief graph and the obtained natural language preference instructions, high-level exploration plan information is obtained using a pre-trained large language model. Based on the local belief graph in the hierarchical belief graph and the high-level exploration plan information, a target node is generated using a pre-trained policy model. Based on the collision-free topology map and the occupied grid map, a collision-free motion trajectory between the current position of the indoor robot and the target node is generated; Based on the collision-free motion trajectory, the indoor robot is controlled to perform inspection operations.

2. The method according to claim 1, wherein, The method further includes: In response to the occupation of the grid map satisfying the preset replanning trigger condition, an updated hierarchical belief map is generated based on the new sensor data collected during the inspection operation. In response to the updated hierarchical belief graph meeting preset conditions, the updated hierarchical belief graph is used as the hierarchical belief graph, and the control steps are continued.

3. The method according to claim 1, wherein, The real-time construction of an indoor environment occupancy grid map based on sensor data collected during the robot's movement includes: The sensor data and the current pose data of the indoor robot are converted into standardized observation data in the world coordinate system; The pre-stored occupied grid map is updated based on the standardized observation data to obtain the occupied grid map.

4. The method according to claim 1, wherein, The generation of a collision-free topology map based on the occupied grid map includes: Extract passable areas from the occupied grid map; Uniform sampling is performed within the passable area to generate a candidate viewpoint set; Determine the nearest neighbor nodes of the target number for each candidate viewpoint in the candidate viewpoint set to generate an initial topology graph; Collision detection is performed on each candidate edge in the initial topology graph to generate a first collision detection result; Based on the first collision detection result, the candidate edges of the initial topology graph are filtered to obtain a filtered candidate edge set; Based on the candidate viewpoint set and the filtered candidate edge set, a collision-free topology graph is generated.

5. The method according to claim 1, wherein, Based on the global belief map in the hierarchical belief map and the acquired natural language preference instructions, a pre-trained large language model is used to obtain high-level exploration plan information, including: The exploration state information of each node, each edge, and each node in the global belief graph is transformed into a structured text description. The natural language preference instructions, the structured text description, and the acquired historical exploration information are combined to generate the input information for the large language model, so as to obtain the input sequence; The large language model is used to iteratively infer the input sequence to generate the original output sequence. The original output sequence is decoded to generate a decision marker and a higher-level access sequence; In response to the ruling marker indicating that reasoning is complete, the high-level access sequence is extracted to obtain candidate high-level exploration plan information; The candidate high-level exploration plan information is mapped to high-level exploration plan information with regional priority and access order.

6. The method according to claim 1, wherein, The generation of target nodes based on the local belief graph in the hierarchical belief graph and the high-level exploration plan information, using a pre-trained policy model, includes: For each node in the local belief graph, a node feature vector is constructed to generate a node feature vector set; The high-level exploration plan information is encoded into guiding features associated with each node of the local belief graph; The node feature vector set and the guiding feature vector are fused to generate fused node feature information; The adjacency relationship between the fused node feature information and the local belief graph is transformed into structured input information; The structured input information is input into the pre-trained policy model to generate priority information for each candidate node; The target node is determined based on the priority information of each candidate node.

7. An indoor robot inspection device based on a large language model, comprising: The building unit is configured to, in response to the indoor robot starting an inspection task, construct an occupied grid map of the indoor environment in real time based on sensor data collected during the movement of the indoor robot. The generation unit is configured to generate a collision-free topology map based on the occupied grid map; The partitioning unit is configured to partition the collision-free topology graph into a global belief graph and a local belief graph to construct a hierarchical belief graph; The control unit is configured to perform the following control steps for the hierarchical belief map: Based on the global belief graph in the hierarchical belief graph and the obtained natural language preference instructions, high-level exploration plan information is obtained using a pre-trained large language model. Based on the local belief graph in the hierarchical belief graph and the high-level exploration plan information, a target node is generated using a pre-trained policy model. Based on the collision-free topology map and the occupied grid map, a collision-free motion trajectory between the current position of the indoor robot and the target node is generated; Based on the collision-free motion trajectory, the indoor robot is controlled to perform inspection operations.

8. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images