Spatial reasoning and RAG knowledge enhancement method and system for routing inspection of internal structure of complex equipment

Abstract

The invention provides a spatial reasoning and RAG knowledge enhancement method and system for routing inspection of an internal structure of complex equipment, and the method comprises the steps: extracting topological knowledge represented in a triple form from equipment design drawings and regulations, converting the extracted triple into a natural language text in a cue card form, constructing a unique vector index through LlamaIndex, and carrying out the routing inspection of the internal structure of the complex equipment; forming a special knowledge base; a small language model platform based on local Olma is built, a related prompt card is automatically called as input when the model executes a reasoning task, the chained reasoning process of a large model is simulated, especially for spatial structure characteristics of a water turbine, the spatial structure of an inner bore of the water turbine generator and the confidence coefficient of the spatial structure are finally reasoned, and the reasoning accuracy of the water turbine generator is improved. And a spatial distribution state of the key component and a corresponding inspection strategy are given.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of industrial intelligent inspection and large language model application, and particularly relates to a space reasoning and RAG knowledge enhancement method and system for complex equipment internal structure inspection. BACKGROUND

[0002] As the core power equipment of a hydropower station, a hydro-generator undertakes the key task of converting mechanical energy of water flow into electrical energy. The inner bore structure of the hydro-generator is extremely precise and compact, and usually includes insulators, excitation windings, cooling channels, and various types of support brackets and other key components. These components are not only densely distributed, but also often in harsh environments such as high temperature, high humidity, and strong electromagnetic interference, making the operation state monitoring and daily inspection work challenging. For a long time, the conventional inspection of a hydropower station mainly relies on technical personnel to personally enter the inner bore of the unit to carry out work, which not only has a small working space and obstructed vision, but also has safety hazards such as high altitude, high voltage, and residual rotating parts. In addition, manual recording and subjective judgment can lead to inconsistent data, high omission rate, and other problems, which seriously affect the comprehensive understanding of the equipment state and the fault warning capability.

[0003] Although the traditional manual inspection mode has played an important role in history, its drawbacks have become increasingly prominent. On the one hand, personnel entering the inner part of the unit need to perform strict power-off, isolation, and safety protection processes, which are time-consuming and inefficient, and are difficult to meet the requirements of modern hydropower stations for operation and maintenance response speed. On the other hand, the detection results are heavily dependent on individual experience, and the judgment standards between different technical personnel are difficult to unify, which can easily overlook early-stage hidden dangers. With the development of hydropower units towards large capacity and complexity, the traditional method has been difficult to achieve fine and systematic control of the equipment state, and it is urgent to introduce automation and intelligent means to improve the inspection quality and coverage.

[0004] In recent years, robots equipped with visual sensors or 3D laser radars have been widely used in industrial detection, and have also provided a new technical path for hydro-generator inner bore inspection. Such robots can replace manual work to enter high-risk or closed spaces, collect high-definition images and three-dimensional point cloud data, and then build a digital model of the inner part of the unit. However, the internal structure of the hydro-generator has highly unstructured characteristics, and the spatial topological relationship between components is complex and severely occluded, which poses high requirements on the perception and navigation capabilities of the robot. The robot not only needs to locate its position in real time, but also needs to understand the three-dimensional spatial relationship between components, and on this basis make reasonable path planning and inspection sequence decisions, otherwise it is easy to produce detection blind spots.

[0005] In processing such tasks with complex three-dimensional structure understanding and multi-step reasoning requirements, mainstream large language models (>70B) exhibit strong chain reasoning and context modeling capabilities, enabling spatial relationship deduction combined with device structure knowledge to assist in generating inspection strategies. However, in industrial field environments, due to considerations of computational resources, response delays, and data privacy protection, it is often impossible to deploy such large models. Most water and power enterprises prefer to deploy lightweight models (such as 7B) on local or edge sides to balance efficiency and security.

[0006] However, small parameter models often perform poorly in understanding professional domain knowledge such as "three-dimensional topological relationships" and "device function and structure associations" due to limited capacity and training data, and are prone to information omission or logical errors. For example, when answering questions such as "how to inspect the excitation winding blocked by the support", the model not only needs to know the component name, but also needs to have spatial orientation reasoning ability and common sense of inspection procedures, and small models often struggle to build such complex context associations independently, resulting in incomplete or deviated outputs. SUMMARY

[0007] The present application provides a spatial reasoning and RAG knowledge enhancement method and system for complex equipment internal structure inspection to address the shortcomings of small language models (<8B) in terms of insufficient computing power and reasoning ability. The method translates triple information into natural language and uses Ollama x LlamaIndex to build an index to answer related questions. The invention improves the reasoning ability of large language models and enhances the performance of local deployment of language models. To achieve the above technical features, the purpose of the present application is realized as follows: a spatial reasoning and RAG knowledge enhancement method for complex equipment internal structure inspection, comprising the following steps: Step one: extract the spatial attribute information of each functional component and internal functional area of the generator from the design drawings and operation procedures of the hydro-generator; convert the spatial attribute information into topological knowledge text representation in the form of triples after semantic analysis; and form a structured knowledge text for spatial reasoning; Step two: convert the structured knowledge text into prompt card natural language text in a preset template format, with each prompt card describing a spatial relationship or component association; store the prompt cards in a local knowledge base as searchable knowledge units for language model invocation; Step three: import the prompt card knowledge base into the RAG system based on LlamaIndex, index the prompt card, and generate unique index identifiers corresponding to each spatial relationship; the RAG system is combined with a lightweight language model with a parameter amount of not more than 7B, which is used to perform external knowledge retrieval and supplement when the language model itself is insufficient in reasoning ability; Step four: when the console inputs an inspection question or a spatial structure query, the language model automatically calls the prompt card associated with the question and obtains the relevant spatial relationship through the RAG mechanism; the model simulates chain spatial logical reasoning in the reasoning process and outputs the natural language description of the position relationship and connection structure of the inner cavity components of the hydroelectric generator.

[0008] Preferably, the spatial attribute information in step one includes component name, spatial position, connection relationship and structure feature.

[0009] Preferably, the prompt card adopts a "one card one relationship" organization method, and each prompt card corresponds to only one topological triple.

[0010] Preferably, the RAG system includes: (1) a semantic vector index module based on LlamaIndex; (2) an Ollama-based lightweight model calling interface; (3) and a knowledge retrieval and chain reasoning scheduling module; the RAG system automatically adjusts the retrieval depth and the number of prompt cards according to the question semantics.

[0011] Preferably, the lightweight language model adopts Llama, Ollama or equivalent model.

[0012] Preferably, the topological knowledge text further contains spatial hierarchy constraint information or relative orientation label.

[0013] Preferably, the language model forms a multi-step reasoning chain by recursively calling multiple prompt cards, and realizes hierarchical generation of the relationship of the internal components of the hydroelectric generator, including but not limited to.

[0014] Another aspect of the present application provides a spatial reasoning and RAG knowledge enhancement system for complex equipment internal structure inspection, which is used to realize the method and includes: A triple knowledge source construction module is used to abstract the inner cavity structure of the hydroelectric generator into triples; A prompt card generation module is used to convert triple knowledge into "prompt card" text conforming to the understanding habit of the small model, using a natural language template; A RAG retrieval module is used to index the prompt card and the inspection document to support the small model to retrieve relevant content; A reasoning question and answer module is used to inject the retrieved prompt card content into the context of the small model to realize "repetition type" spatial reasoning and answering. The robot inspection decision module generates a next observation point according to the inference result output by the small model.

[0015] Another aspect of the present application provides a construction method of a spatial reasoning and RAG knowledge enhancement system for complex equipment internal structure inspection, comprising the following steps: S1, knowledge template extraction: extracting the equipment structure relationship from design drawings and maintenance regulations by artificial or semi-automatic means to form a standardized triple set; S2, prompt card: using rule templates to convert the triple set into concise and readable natural language prompt cards, maintaining the principle of "one card one relationship"; S3, RAG construction: importing the prompt card into LlamaIndex to establish a vector index and deploying it in the Ollama small model backend; S4, dynamic injection and retrieval: during reasoning, the system queries the relevant prompt card according to the question and injects the model context; S5, spatial reasoning output: the small model implements basic spatial structure understanding and answering by repeating and splicing prompt card information; S6, task planning and inspection feedback: outputting task planning according to the reasoning result and forming a closed loop through robot execution and on-site verification.

[0016] The present application has the following advantages: 1. The present application provides a feasible technical path for making up for the short board of small models in professional field semantic understanding and spatial reasoning, which is to adopt the strategy of combining retrieval enhancement generation (RAG) and structured knowledge injection. By constructing a knowledge base specific to hydroelectric generating sets, heterogeneous information such as equipment drawings, three-dimensional model descriptions, and historical inspection records is stored in vectors, enabling the model to retrieve relevant context in real time when answering questions; at the same time, by injecting equipment structure knowledge, spatial relationship terminology, and inspection procedures, the model's semantic understanding ability in professional scenarios is enhanced. In this way, even in resource-constrained environments, small models can have preliminary spatial reasoning and inspection question and answer capabilities, thereby supporting the intelligent transformation of hydroelectric operation and maintenance.

[0017] 2. The present application provides a spatial reasoning and RAG knowledge enhancement method for complex equipment internal structure inspection, which is a knowledge injection method based on an external RAG knowledge base. This method translates triple information into natural language and calls information to answer related questions through Ollama x LlamaIndex index construction. This invention improves the problem that the reasoning ability of large language models cannot be realized on small language models, and improves the performance of local deployment of language models.

[0018] 3、The application converts the triple information into a prompt card in a specific format, which can accurately extract knowledge from the language model while retaining the readability of the information, facilitating the maintenance of the knowledge base or the addition of new knowledge base, and providing a channel for post-production supplement for the model.

[0019] 4、The application adopts a "one card and one relationship" organization method, and each prompt card corresponds to only one topological triple, so as to reduce the calculation complexity of model reasoning and improve the index accuracy.

[0020] 5、The application can improve the spatial reasoning accuracy of the language model in the complex internal structure by containing spatial hierarchical constraint information or relative orientation labels in the topological knowledge text. BRIEF DESCRIPTION OF DRAWINGS

[0021] The application will be further described below in combination with the drawings and examples.

[0022] Figure 1 The specific implementation process of the method is shown, which includes extracting related knowledge from the water turbine, then indexing the knowledge through RAG and generating a prompt card, when the user queries on the host computer, the language model can dynamically retrieve the knowledge base and generate an answer.

[0023] Figure 2 The structure of the triple knowledge is shown, mainly including two objects and their relationship, and the parent object and the child object connected by the arrow indicating the relationship.

[0024] Figure 3 The overall flow of RAG retrieval is shown, the chart on the left upper side shows the processing process of RAG for the prompt card, and the right lower side shows how the RAG system matches the user's question with the keywords in the knowledge base and gives an answer when the user queries. DETAILED DESCRIPTION

[0025] The application will be further described in detail below through specific examples. The following examples can enable a person skilled in the art to more fully understand the application, but do not limit the application in any way.

[0026] I、The overall flow of the method of the application The application provides a spatial reasoning and RAG knowledge enhancement method for complex equipment internal structure inspection, and the overall flow includes the following steps: Knowledge extraction: extracting key components and their spatial relationships from the water turbine generator design drawings, maintenance procedures and field annotation data to construct a standardized triple knowledge set.

[0027] Prompt card generation: converting triple knowledge into natural language description in the form of a prompt card for the model to quickly understand.

[0028] RAG Indexing: Indexing prompt cards using LlamaIndex and accessing small language models (7B level) run by Ollama.

[0029] Dynamic retrieval and injection: When generating reasoning questions or inspection strategies, retrieve relevant prompt cards based on user questions or robot status and inject model context.

[0030] Spatial reasoning and output: Small models complete spatial reasoning based on injected prompt cards and give answers or generate next step inspection strategies.

[0031] Closed-loop execution and feedback: Convert model output into robot instructions, execute inspection in internal environment, and return results to knowledge base to update prompt cards.

[0032] II. Example 1: Construction of triple knowledge source Take a certain type of hydro-generator as an example, its internal structure includes: rotor, stator core, excitation winding, insulator, cooling air duct, support seat, observation port, etc.

[0033] (1) Knowledge extraction rules: Extract component position relationships from two-dimensional design drawings, for example: The stator core is located outside the rotor; The cooling air duct is distributed around the stator core; Extract operation and detection point information from maintenance procedures, for example: "Take a photo of the cooling air duct to check for dust accumulation"; "Check if there are cracks on the surface of the rotor magnetic pole"; (2) Triple sample: Convert the above relationships to a unified format:

[0034] This triple data is saved in a structured file (such as CSV or relational database) for easy automatic updating.

[0035] III. Example 2: Prompt card generation and organization Due to the limited reasoning ability of small language models, the invention designs a prompt card (Prompt Card) form of knowledge injection strategy to convert complex topological knowledge into natural language short sentences that are easy for models to understand.

[0036] (1) Prompt card format Use a fixed template:

Prompt Card

[0037] IV. Example 3: RAG Retrieval and Reasoning Process (1) Search trigger When the user or host computer raises a question, such as: Where is the stator base located? The system will automatically call the RAG module to retrieve knowledge from the knowledge base.

[0038] (2) Injection and Reasoning The retrieved relevant cue cards are injected into the context of a small language model, for example: Model input: Question: Where is the stator base located? Context: - Tip Card 1: The stator base is located above the foundation pit, and its lower part is connected to the wheel axle; - Prompt card 3: The stator is located inside the stator base, and the rotor is inside it. Model output (spatial reasoning): "The stator base is above the foundation pit, outside the stator." Five, Example 4: Closed-loop execution and knowledge update Task execution: The robot enters the inner bore according to the model reasoning scheme, completes image acquisition and defect detection.

[0039] Result feedback: If new structural features or obstacle positions are found on site that are inconsistent with the original triplets, update the triplet knowledge base through manual correction or automatic labeling.

[0040] Prompt card update: Automatically generate or revise prompt cards based on new knowledge and re-index them to ensure the timeliness and accuracy of model reasoning.

[0041] Loop iteration: The system continuously optimizes the knowledge source based on field experience, enhancing the coverage of model reasoning.

[0042] Six, The system of the present invention includes the following modules: 1. Triplet knowledge source construction module: Abstract the inner bore structure of the hydroelectric generator as triplets: (Component A, Relationship R, Component B), for example: (Stator core, wrapped around, rotor) (Cooling air duct, located outside, stator core) (Observation point, corresponding, key defect location) 2. Prompt card generation module: Convert triplet knowledge into "prompt card" text that conforms to the habits of small models, using natural language templates, for example: "The stator core is outside the rotor. To observe the surface of the stator core, you should choose an observation point located outside the rotor." 3. RAG retrieval module (LlamaIndex × Ollama): Index the prompt cards and inspection documents to support small model retrieval of related content.

[0043] 4. Reasoning question and answer module: Inject the retrieved prompt card content into the model's context to achieve "parrot-style" spatial reasoning answers.

[0044] 5. Robot inspection decision module: Generate the next observation point based on the model output reasoning result.

[0045] Although the preferred embodiments of the present application have been described above with reference to the accompanying drawings, the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative and are not restrictive. A person of ordinary skill in the art can make many specific changes to the present application without departing from the spirit of the present application and the scope of protection of the claims, and these all belong to the scope of protection of the present application.

Claims

1. A spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment, characterized in that, Includes the following steps: Step 1: Extract spatial attribute information of each functional component and internal functional area of ​​the hydro-generator from the design drawings and operating procedures; convert the spatial attribute information into a topological knowledge text representation in the form of triplets after semantic parsing, so as to form a structured knowledge text for spatial reasoning. Step 2: Convert the structured knowledge text into natural language text for prompt cards that conform to a preset template format. Each prompt card describes a spatial relationship or component association. The prompt cards are stored in the local knowledge base and serve as searchable knowledge units when the language model is invoked. Step 3: Import the cue card knowledge base into the RAG system built on LlamaIndex, index the cue cards, and generate unique index identifiers corresponding to each spatial relationship; the RAG system is combined with a lightweight language model with no more than 7B parameters to perform external knowledge retrieval and supplementation when the reasoning ability of the language model itself is insufficient. Step 4: When an inspection question or spatial structure query is entered into the console, the language model automatically calls the prompt card associated with the question and obtains the relevant spatial relationships through the RAG mechanism; During the reasoning process, the model simulates chain-like spatial logic reasoning and outputs a natural language description of the positional relationships and connection structures of the internal components of the hydro-generator.

2. The spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 1, characterized in that, The spatial attribute information in step one includes component name, spatial location, connection relationship, and structural features.

3. The spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 1, characterized in that, The prompt cards are organized in a "one card, one relationship" manner, with each prompt card corresponding to only one topological triplet.

4. The spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 1 or 2, characterized in that, The RAG system includes: (1) a semantic vector indexing module based on LlamaIndex; (2) a lightweight model calling interface based on Ollama; and (3) a knowledge retrieval and chain reasoning scheduling module. The RAG system automatically adjusts the retrieval depth and the number of calling prompt cards according to the semantics of the question.

5. The spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 4, characterized in that, The lightweight language model uses Llama, Ollama, or an equivalent model.

6. The spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 1 or 2, characterized in that, The topological knowledge text further includes spatial hierarchy constraint information or relative orientation labels.

7. A spatial reasoning and RAG knowledge enhancement method for inspecting the internal structure of complex equipment according to claim 1 or 2, characterized in that, The language model forms a multi-step reasoning chain by recursively calling multiple cue cards, enabling hierarchical generation of relationships between internal components of a hydro-generator, including but not limited to those between components.

8. A spatial reasoning and RAG knowledge enhancement system for inspecting the internal structure of complex equipment, characterized in that, The system is used to implement the method according to any one of claims 1-7, comprising: The ternary knowledge source construction module is used to abstract the internal structure of a hydro generator into a ternary; The cue card generation module is used to convert triple knowledge into "cue card" text that conforms to the understanding habits of small models, using natural language templates; The RAG retrieval module is used to index the prompt cards and inspection documents to support small models in retrieving relevant content; The reasoning question-answering module is used to inject the retrieved prompt card content into the context of the small model to achieve "repetition-style" spatial reasoning answers; The robot inspection decision module generates the next observation point based on the reasoning results output by the small model.

9. The method for constructing a spatial reasoning and RAG knowledge enhancement system for inspecting the internal structure of complex equipment as described in claim 8, characterized in that, Includes the following steps: S1, Knowledge Template Extraction: Extracting equipment structural relationships from design drawings and maintenance procedures through manual or semi-automatic methods to form a standardized set of triples; S2, cue card formatting: Using rule templates, the set of triples is transformed into concise and readable natural language cue cards, maintaining the principle of "one card, one relationship"; S3, RAG Construction: Import the cue cards into LlamaIndex to create vector indices and deploy them on the Ollama small model backend; S4, Dynamic Injection and Retrieval: During inference, the system queries relevant hint cards based on the question and injects them into the model context; S5, Spatial Reasoning Output: The small model achieves basic spatial structure understanding and answering by restating and piecing together information from the cue cards; S6, Task Planning and Inspection Feedback: Outputs task planning based on reasoning results, and forms a closed loop through robot execution and on-site verification.

Images