A heterogeneous multi-robot task allocation method and system based on large language models and RAG

Abstract

This invention discloses a heterogeneous multi-robot task allocation method and system based on a large language model and RAG (Real-Age Graph), used for task allocation in emergency rescue operations. First, knowledge graph technology is used to structurally represent the characteristics of various robot types, resulting in robot profiles. Second, all robot profiles, current real-time environmental data, and a database of historical rescue cases similar to the current natural disaster type are segmented and vectorized to construct a vector database. In the task allocation stage: first, the requirements description of the rescue task is vectorized into query vectors, and similarity retrieval is performed in the vector database to obtain a preset number of knowledge fragments. Then, the obtained knowledge fragments and the requirements description of the rescue task form an enhanced context, which is then optimized through prompting engineering and input into the large language model to allocate the rescue task to various robot individuals. This invention enables accurate matching and dynamic allocation of emergency rescue tasks.

Description

Technical Field

[0001] This invention belongs to the field of collaborative task allocation, and in particular relates to a heterogeneous multi-robot task allocation method and system based on a large language model and RAG. Background Technology

[0002] Due to the unpredictability of disaster environments and the diversity of mission requirements, disaster sites such as earthquakes and floods often require the collaborative operation of various heterogeneous robots, including aerial drones, underwater robots, land search robots, and medical transport robots. These robots each have their own characteristics in terms of function, performance, and environmental adaptability, while rescue missions have varying degrees of urgency, complexity, and professional requirements. This makes the efficient and rational allocation of rescue tasks a critical issue that urgently needs to be addressed.

[0003] Current heterogeneous robot task allocation technologies primarily employ traditional optimization methods, namely mathematical modeling, to transform the task allocation problem into a linear programming or integer programming problem, seeking the optimal solution by defining an objective function and constraints. These methods include dynamic programming and heuristic optimization. While these methods have a rigorous mathematical foundation in theory, they reveal significant shortcomings in practical emergency rescue scenarios. They heavily rely on pre-defined parameters and rules, often requiring remodeling and recalculation when encountering changes in robot performance or unexpected tasks, hindering rapid response. As the problem scale increases, the computational complexity of mathematical programming methods rises sharply, often requiring several minutes or even longer to provide a solution, failing to meet the time-sensitive demands of emergency rescue. For example, in mountain rescue operations, sudden weather changes may alter drone flight parameters, making it difficult for traditional optimization models to adjust task allocation strategies in a timely manner.

[0004] Meanwhile, existing technologies suffer from serious deficiencies in knowledge utilization. Traditional methods primarily rely on numerical performance parameters, neglecting the valuable domain knowledge embedded in unstructured texts such as technical documents and operation manuals. For example, empirical knowledge such as "the detection accuracy of underwater robots decreases in turbid waters" is difficult to fully express using simple numerical parameters. Summary of the Invention

[0005] This invention provides a heterogeneous multi-robot task allocation method and system based on a large language model and RAG, which can dynamically adjust the allocation strategy, improve the accuracy of emergency rescue task allocation, and effectively meet the timeliness requirements of emergency rescue tasks.

[0006] To achieve the above technical objectives, the present invention adopts the following technical solution:

[0007] A heterogeneous multi-robot task allocation method based on a large language model and RAG is applied to task allocation for emergency rescue missions in current natural disasters, including:

[0008] Knowledge graph technology is used to structurally represent the functional characteristics of various types of robots, which are recorded as profiles of each type of robot.

[0009] All types of robot profiles, current real-time environmental data, and historical rescue case databases of the same type as the current natural disaster are segmented and vectorized to construct a vector database;

[0010] In the task allocation phase: First, the requirements description of the rescue task is vectorized into query vectors, and similarity retrieval is performed in the vector database to obtain a preset number of knowledge fragments; then, the obtained knowledge fragments and the requirements description of the rescue task are combined to form an enhanced context, and after prompting engineering optimization, they are input into the large language model. The large language model outputs the allocation scheme of the rescue task to various robot individuals.

[0011] Furthermore, the robot's functional characteristics are obtained by extracting entities and relationships from the robot's technical documents and historical rescue cases using a large language model and introducing a thought chain approach.

[0012] Furthermore, knowledge graph technology is used to structurally represent the functional characteristics of any type of robot, including:

[0013] First, a large language model is used to extract preset performance parameters from the robot's technical documentation;

[0014] Secondly, key entities in the core performance parameters of the robot are extracted based on a large language model;

[0015] Secondly, by guiding the large language model to perform in-depth reasoning, logical relationships between different entities are identified; when logical relationships exist between entities, the corresponding triples are extracted.

[0016] Then, the large language model is guided to verify the logical relationship of the extracted triples and output the triples that actually have logical relationships.

[0017] Finally, a knowledge graph is generated based on the output triples, which is the robot profile.

[0018] Furthermore, the key entities include: entities related to motion performance, entities related to perception performance, and entities related to operational performance.

[0019] Furthermore, the real-time environmental data, including meteorological monitoring data and topographic data of the current time and location of natural disasters, together form structured data.

[0020] Furthermore, a Markdown text splitter is used to employ corresponding chunking methods for different types of data.

[0021] Furthermore, through the BGE-M3 embedding model, each data block is mapped to a high-dimensional space to obtain semantic vectors, and all semantic vectors constitute a vector database.

[0022] Furthermore, the BGE-M3 embedding model is used to vectorize the requirements of the rescue mission into query vectors.

[0023] A heterogeneous multi-robot task allocation system based on a large language model and RAG is proposed, which uses the aforementioned heterogeneous multi-robot task allocation method to allocate tasks for current natural disaster relief missions.

[0024] This invention introduces a Large Language Model (LLM), which, based on its powerful semantic understanding capabilities, can deeply analyze the language requirements expressed in emergency rescue mission descriptions and accurately understand complex instructions such as "prioritize the transfer of seriously injured personnel," thereby automatically identifying the core requirements and complexity of the mission.

[0025] In the robot profiling stage, a large language model based on thought chains is used to generate heterogeneous robot profiles, characterizing the key capabilities of each robot. In the knowledge base construction and retrieval process, an emergency rescue multimodal knowledge base is generated through robot profiles, real-time environmental sensor data, and historical rescue cases. Similarity retrieval is performed using text vectorization, which is then combined with task queries to form detailed contextual information. This significantly reduces the "illusion" phenomenon of large language models in professional rescue fields, enabling accurate matching and dynamic allocation of emergency rescue tasks.

[0026] Therefore, this invention can automatically identify the core needs and complexities of emergency rescue missions, providing rescue solutions that meet actual requirements for different missions. By applying knowledge graphs and Retrieval-Augmented Generation (RAG) technologies, it reduces the reasoning error rate of large language models and improves the accuracy of rescue mission allocation through dynamic knowledge base adaptation, while significantly reducing emergency rescue response time and enhancing the model's dynamic scenario adaptability. Attached Figure Description

[0027] Figure 1 This is a framework diagram of the heterogeneous multi-robot task allocation method described in an embodiment of the present invention.

[0028] Figure 2 This is the thought chain reasoning for entity-relation extraction described in the embodiments of the present invention. Detailed Implementation

[0029] The embodiments of the present invention will be described in detail below. These embodiments are based on the technical solutions of the present invention and provide detailed implementation methods and specific operation processes to further explain the technical solutions of the present invention.

[0030] This invention provides a heterogeneous multi-robot task allocation method based on a large language model and RAG (Related Aspects of Engineering). This method addresses issues such as long response time, low knowledge utilization, and poor adaptability to dynamic scenarios in robot task allocation during emergency rescue. First, a robot profile based on a knowledge graph is established to accurately model the robot's functional characteristics. Second, a comprehensive multimodal rescue knowledge base is constructed by combining the robot profile, real-time environmental data (such as disaster type, terrain complexity, and weather conditions), and a historical rescue case database. Then, the knowledge base content is transformed into semantic vectors using a BGE-M3 embedding model, establishing a FAISS vector database to support efficient retrieval. Finally, the retrieved relevant knowledge fragments are used as context input, optimized through prompting engineering, and then input into the large language model to generate the final task allocation scheme. Using the RAG method to optimize the heterogeneous robot task allocation strategy can reduce the reasoning error rate of the large language model and improve allocation accuracy through dynamic knowledge base adaptation, thereby increasing the rescue speed in appropriate scenarios. Figure 1 As shown.

[0031] Step 1: Use knowledge graph technology to structurally represent the functional characteristics of various types of robots, and record them as the profiles of each type of robot.

[0032] Using a large language model, key information is extracted from technical documents reflecting the robot's functional characteristics. This includes core parameters such as mobility (e.g., maximum flight altitude of a drone, obstacle-crossing capability of a ground robot), perception capabilities (e.g., infrared detection range, underwater sonar accuracy), and operational capabilities (e.g., robotic arm payload, medical cabin capacity). A thought chain reasoning method ensures the accuracy of entity recognition and relation extraction. For example, for an underwater rescue robot, the system records key performance indicators such as its maximum diving depth and the attenuation coefficient of detection accuracy in turbid water, as well as historical data on collaboration with other robots. This information is organized into triplets in the form of (subject, predicate, object) to construct a robot profile encompassing functional, environmental adaptation, and collaborative dimensions.

[0033] Define heterogeneous robot swarms as ,in Indicates the first Robot-like. (Reference) Figure 2 As shown, a robot is constructed using the LLM (Leadership in Modeling) approach to introduce thought chains. The process of creating a portrait includes the following steps:

[0034] First, LLM is used to extract core performance parameters from the robot's technical documentation. Let the original text be represented as... After processing, the extracted core performance parameters can be expressed as follows: .

[0035] ;

[0036] Secondly, core performance parameters of the robot are extracted based on LLM. Key Entity Set Specifically, when analyzing the core performance parameters of a robot, LLM defines key capability indicators that can represent the robot's characteristics as entities:

[0037] ;

[0038] For example, typical entity types include:

[0039] Motion performance entity: ;

[0040] Perceived performance entities: .

[0041] Secondly, by guiding the large language model to perform in-depth reasoning, logical relationships between different entities are identified; when logical relationships exist between entities, the corresponding triples are extracted. Based on this, basic units in a knowledge graph are generated, with entities as nodes and relationships as connecting edges, forming... Semantic structure of form.

[0042] ;

[0043] in, For the set of extracted triples, and They are from the entity set respectively The head entity and the tail entity, It is a set of relations.

[0044] Then, the large language model is guided to reflect on and self-evaluate the extracted triples, that is, to verify the logical relationships and output the triples that actually have logical relationships.

[0045] Finally, regarding robots Generate robot profiles based on extracted triples ,in Represents a set of entities. This is a set of relationships used to achieve a structured representation of robot information. The final robot profile is stored in the Neo4j knowledge graph database.

[0046] Step 2: For all types of robot profiles, current real-time environmental data, and historical rescue case databases of the same type of natural disaster, perform block-based and vectorized representations respectively to construct a vector database.

[0047] Step 2.1: Construct a knowledge base using a multi-source heterogeneous data acquisition strategy.

[0048] The knowledge base construction process employs a multi-source data fusion strategy, integrating three core data types: a robot profiling knowledge graph, real-time environmental sensor data (including structured data such as meteorological monitoring and terrain scanning of the disaster location at the current time), and a historical rescue case database (containing textual materials such as task logs and action reports of the same disaster type from the past 30 years). All data undergoes unified cleaning and standardization processing before serving as the source for the rescue knowledge base. The rescue knowledge base is represented as follows:

[0049] ;

[0050] in, Documents that form part of the rescue knowledge base. This refers to a set of robot portraits, i.e. . It represents structured data, such as real-time environmental sensor data. It represents unstructured text, covering historical rescue case databases, mission logs, rescue reports, etc.

[0051] Step 2.2, knowledge base corpus segmentation.

[0052] The document is intelligently segmented using the Markdown text splitter provided by LangChain. Targeted document slicing methods are employed based on different data characteristics. For example, spatial information such as terrain scan data is sliced ​​using geographic grid encoding; text data such as task logs are segmented while maintaining semantic integrity. For the document... The slicing function is:

[0053] ;

[0054] in, , and These are knowledge data blocks corresponding to robot profiles, structured data, and text data, respectively.

[0055] Step 2.3: Generate semantic vectors.

[0056] The semantic information of the text is captured using the BGE-M3 embedding model, and the text is transformed into a numerical vector representation. This representation is then stored in the FAISS vector database with an efficient index.

[0057] For knowledge data segmentation Embedded models will Mapping to a high-dimensional vector space, the mapping function is: ,in This represents the dimension of the vector space. The generated semantic vector is represented as:

[0058] ;

[0059] Using text vectors Summarize the semantic features of the original text and store them in the FAISS vector database. In this process, an index is created in the FAISS vector database to index the stored semantic vectors, supporting subsequent text semantic similarity analysis and retrieval.

[0060] Step 3, Task Allocation Phase.

[0061] Step 3.1: Vectorize the description of the rescue mission requirements (such as "prioritize the transfer of seriously injured patients in the western district") into a query vector.

[0062] For task query Using the BGE-M3 embedding model The query vector is generated by mapping the user's input question text to a high-dimensional vector space. .

[0063] ;

[0064] in, It is the target space of the embedded vectors. It is the vector dimension.

[0065] Step 3.2: Perform similarity retrieval in the vector database to obtain a preset number of knowledge fragments; the retrieval process in this embodiment uses cosine similarity calculation.

[0066] In FAISS Vector Database Perform a similarity search to find results matching the query vector. The most similar knowledge data is segmented. This process relies on calculating the similarity between two vectors.

[0067] ;

[0068] in, This refers to data stored in the FAISS vector database. The knowledge base text vector in the text.

[0069] Step 3.3: Sort the retrieved knowledge fragments based on vector similarity and select the top ones based on similarity ranking. These fragments, which may include matching robot performance parameters, historical task records in similar environments, etc., will serve as the final search results. .

[0070] Step 3.4, obtain A knowledge fragment With task query (The requirements description for the rescue mission) together constitute an enhanced context for fusion retrieval of knowledge. .

[0071] Step 3.5: Design a structured prompting engineering template to guide model reasoning, enhancing the context. Combined with prompting project The input is fed into a large language model for deep reasoning. The large language model outputs an allocation scheme for rescue tasks to various types of individual robots, along with the reasons for the allocation and the related knowledge base sources, ensuring the transparency and credibility of the output results.

[0072] As shown in Table 1, the prompt engineering template explicitly requires the large language model to comprehensively consider factors such as the robot's real-time status, environmental constraints, and task priorities, outputting a specific allocation scheme for each robot, along with an explanation of the allocation basis. For example, in a mountain rescue scenario, the system might recommend a drone to perform high-altitude reconnaissance missions, while designating a certain type of ground robot to be responsible for transporting the wounded, and detailing how the robot's anti-slip performance and load-bearing capacity meet the transportation needs in the current muddy terrain.

[0073] ;

[0074] This invention employs a dynamic update mechanism, with robot status sensors continuously uploading the latest data and new rescue cases being periodically added to the knowledge base. When sudden environmental changes occur (such as a sudden downpour causing reduced visibility), the system can quickly reassess the task allocation plan, ensuring the timeliness of task assignment. The application of RAG technology significantly reduces the "illusion" phenomenon of large language models in professional rescue fields, improving the model's allocation accuracy. Simultaneously, it effectively meets the timeliness requirements of emergency rescue, thereby increasing the speed of rescue in appropriate scenarios.

[0075] The above embodiments are preferred embodiments of this application. Those skilled in the art can make various changes or improvements based on them. Without departing from the overall concept of this application, these changes or improvements should fall within the scope of protection claimed in this application.

Claims

1. A heterogeneous multi-robot task allocation method based on a large language model and RAG, characterized in that, It is used for task allocation in emergency rescue operations for current natural disasters, including: Knowledge graph technology is used to structurally represent the functional characteristics of various types of robots, which are recorded as profiles of each type of robot. All types of robot profiles, current real-time environmental data, and historical rescue case databases of the same type as the current natural disaster are segmented and vectorized to construct a vector database; In the task allocation phase: First, the requirements description of the rescue task is vectorized into a query vector, and a similarity search is performed in the vector database to obtain a preset number of knowledge fragments; then, the obtained knowledge fragments and the requirements description of the rescue task are combined to form an enhanced context, and after prompting engineering optimization, they are input into the large language model. The large language model outputs the allocation scheme of the rescue task to various robot individuals. The functional characteristics of the robot were obtained by extracting entities and relationships from the robot's technical documents and historical rescue cases using a large language model and introducing a thought chain approach. Use a Markdown text splitter to employ corresponding chunking methods for different types of data; By using the BGE-M3 embedding model, each data block is mapped to a high-dimensional space to obtain semantic vectors, and all semantic vectors constitute a vector database. By using the BGE-M3 embedding model, the requirements description of the rescue mission is vectorized into a query vector.

2. The heterogeneous multi-robot task allocation method according to claim 1, characterized in that, Knowledge graph technology is used to represent the functional characteristics of any type of robot in a structured manner, including: First, a large language model is used to extract preset performance parameters from the robot's technical documentation; Secondly, key entities in the core performance parameters of the robot are extracted based on a large language model; Secondly, by guiding the large language model to perform in-depth reasoning, logical relationships between different entities are identified; when logical relationships exist between entities, the corresponding triples are extracted. Then, the large language model is guided to verify the logical relationship of the extracted triples and output the triples that actually have logical relationships. Finally, a knowledge graph is generated based on the output triples, which is the robot profile.

3. The heterogeneous multi-robot task allocation method according to claim 2, characterized in that, The key entities include: entities related to motion performance, entities related to perception performance, and entities related to operational performance.

4. The heterogeneous multi-robot task allocation method according to claim 1, characterized in that, The real-time environmental data includes meteorological monitoring data and topographic data of the current time and location of natural disasters, which together form structured data.

5. A heterogeneous multi-robot task allocation system based on a large language model and RAG, characterized in that, The heterogeneous multi-robot task allocation method described in any one of claims 1-4 is used to allocate tasks for the current natural disaster relief mission.

Images